You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@dolphinscheduler.apache.org by Eric Gao <er...@gmail.com> on 2022/06/13 11:56:00 UTC
[PROPOSAL] Improve DolphinScheduler Monitoring
Hi community,
To better match the slogan `Choose good tools, Back home early. Use Right
Scheduler, Sleep Tight `, this mail thread proposes improving the
monitoring of DolphinScheduler.
Currently, in the officially-released versions of DS, there is only
`statistics` feature but no metrics which could be exposed and better
monitored in external systems such the `prometheus + grafana` solution.
Metrics also enable users to better prevent scheduling failures and track
down the bugs when failures happen. We find that DS has already integrated
`micrometers`, therefore, we are working on adding metrics into
DolphinScheduler based on the previous work of
https://github.com/apache/dolphinscheduler/pull/6840
Actually this proposal has been brought up and discussed several times in
community bi-weekly meetings and an initial PR has been submitted by
Wenjun. For details such as progress, action items, etc. please check these
two links: https://docs.qq.com/doc/DTGFiSkRIbHBIeVp3 and
https://github.com/apache/dolphinscheduler/issues/9324
Discussions and suggestions are welcomed and appreciated! (Either by
replying directly to this mail thread or commenting in github issue
https://github.com/apache/dolphinscheduler/issues/9324)
Thanks!
--
Best Regards
Eric Gao
Re: [PROPOSAL] Improve DolphinScheduler Monitoring
Posted by Jiajie Zhong <zh...@gmail.com>.
Agree with wenjun, metric is an important part, especially the
scheduling system, thank both of you to bring it up or start the
metric job
On Mon, Jun 13, 2022 at 10:04 PM Wenjun Ruan <be...@gmail.com> wrote:
>
> +1,
>
> Monitor is important, thanks to Eric working on this, also thanks to
> zhenxu for his previous work.
>
> Thanks,
> Wenjun
>
>
> On Mon, Jun 13, 2022 at 7:56 PM Eric Gao <er...@gmail.com> wrote:
> >
> > Hi community,
> >
> > To better match the slogan `Choose good tools, Back home early. Use Right
> > Scheduler, Sleep Tight `, this mail thread proposes improving the
> > monitoring of DolphinScheduler.
> >
> > Currently, in the officially-released versions of DS, there is only
> > `statistics` feature but no metrics which could be exposed and better
> > monitored in external systems such the `prometheus + grafana` solution.
> > Metrics also enable users to better prevent scheduling failures and track
> > down the bugs when failures happen. We find that DS has already integrated
> > `micrometers`, therefore, we are working on adding metrics into
> > DolphinScheduler based on the previous work of
> > https://github.com/apache/dolphinscheduler/pull/6840
> >
> > Actually this proposal has been brought up and discussed several times in
> > community bi-weekly meetings and an initial PR has been submitted by
> > Wenjun. For details such as progress, action items, etc. please check these
> > two links: https://docs.qq.com/doc/DTGFiSkRIbHBIeVp3 and
> > https://github.com/apache/dolphinscheduler/issues/9324
> >
> > Discussions and suggestions are welcomed and appreciated! (Either by
> > replying directly to this mail thread or commenting in github issue
> > https://github.com/apache/dolphinscheduler/issues/9324)
> >
> > Thanks!
> >
> > --
> > Best Regards
> >
> > Eric Gao
--
Best Wish
— Jiajie
Re: [PROPOSAL] Improve DolphinScheduler Monitoring
Posted by Wenjun Ruan <be...@gmail.com>.
+1,
Monitor is important, thanks to Eric working on this, also thanks to
zhenxu for his previous work.
Thanks,
Wenjun
On Mon, Jun 13, 2022 at 7:56 PM Eric Gao <er...@gmail.com> wrote:
>
> Hi community,
>
> To better match the slogan `Choose good tools, Back home early. Use Right
> Scheduler, Sleep Tight `, this mail thread proposes improving the
> monitoring of DolphinScheduler.
>
> Currently, in the officially-released versions of DS, there is only
> `statistics` feature but no metrics which could be exposed and better
> monitored in external systems such the `prometheus + grafana` solution.
> Metrics also enable users to better prevent scheduling failures and track
> down the bugs when failures happen. We find that DS has already integrated
> `micrometers`, therefore, we are working on adding metrics into
> DolphinScheduler based on the previous work of
> https://github.com/apache/dolphinscheduler/pull/6840
>
> Actually this proposal has been brought up and discussed several times in
> community bi-weekly meetings and an initial PR has been submitted by
> Wenjun. For details such as progress, action items, etc. please check these
> two links: https://docs.qq.com/doc/DTGFiSkRIbHBIeVp3 and
> https://github.com/apache/dolphinscheduler/issues/9324
>
> Discussions and suggestions are welcomed and appreciated! (Either by
> replying directly to this mail thread or commenting in github issue
> https://github.com/apache/dolphinscheduler/issues/9324)
>
> Thanks!
>
> --
> Best Regards
>
> Eric Gao