You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flink.apache.org by Dan Hill <qu...@gmail.com> on 2021/09/22 15:10:05 UTC

Observability tools on top of Flink

Hi!

I saw a recent Medium article
<https://medium.com/pinterest-engineering/faster-flink-adoption-with-self-service-diagnosis-tool-at-pinterest-50a07143f444#id_token=eyJhbGciOiJSUzI1NiIsImtpZCI6ImMzMTA0YzY4OGMxNWU2YjhlNThlNjdhMzI4NzgwOTUyYjIxNzQwMTciLCJ0eXAiOiJKV1QifQ.eyJpc3MiOiJodHRwczovL2FjY291bnRzLmdvb2dsZS5jb20iLCJuYmYiOjE2MzIzMjI1OTUsImF1ZCI6IjIxNjI5NjAzNTgzNC1rMWs2cWUwNjBzMnRwMmEyamFtNGxqZGNtczAwc3R0Zy5hcHBzLmdvb2dsZXVzZXJjb250ZW50LmNvbSIsInN1YiI6IjExNzIzOTM4NzY4MTEyNzQ0NjE4OCIsImhkIjoicHJvbW90ZWQuYWkiLCJlbWFpbCI6ImRoaWxsQHByb21vdGVkLmFpIiwiZW1haWxfdmVyaWZpZWQiOnRydWUsImF6cCI6IjIxNjI5NjAzNTgzNC1rMWs2cWUwNjBzMnRwMmEyamFtNGxqZGNtczAwc3R0Zy5hcHBzLmdvb2dsZXVzZXJjb250ZW50LmNvbSIsIm5hbWUiOiJEYW4gSGlsbCIsInBpY3R1cmUiOiJodHRwczovL2xoMy5nb29nbGV1c2VyY29udGVudC5jb20vYS9BQVRYQUp4c3NCa2k0eklNNXJFXzVZZ0JoVHJ6NlAzVThkWWZoQVZIdkJrPXM5Ni1jIiwiZ2l2ZW5fbmFtZSI6IkRhbiIsImZhbWlseV9uYW1lIjoiSGlsbCIsImlhdCI6MTYzMjMyMjg5NSwiZXhwIjoxNjMyMzI2NDk1LCJqdGkiOiI3MjhjYTFkYjc5ZmVmMDg5NDliZGZiOGNmNTkyNzJhNjEyYWEwYjQyIn0.W-XUXxlly_L4S8OOqQdXd6hnJzWlaA2x_Iy9Uuvl3A-0wc-V-bEGbXxTb0gdhDZHCpzIs6U7wD04Mq1NXMv38FXEyf_BNkzPepby9lwT3CZxCG4XlQp7Eq7a5qrOmXcXZEjteRovJgOcxvJISAbnykXfQdyn0K4JOiT07EE_uOUd56o2V_OGFfxEcBJHMhkXedaICKbczz_-UowNhuwiIInAmhdqW9ZXo21_oBwWQCDKQL4vFgU-sWwxi2b-oa7bhzqNMP4dcWf0r8vvgxR4W4RqAI2oGvBAbE0-eCL8QvL0GPpi0t1tHl5Y57UG6PBvmP66cI2_OsWmN5GHTttB7g>
by Pinterest on a diagnostic tool called DrSquirrel.  Looks great.

What do other teams do for similar problems?

E.g. Has anyone piped their Flink metadata into Apache Thirdeye?

- Dan

Re: Observability tools on top of Flink

Posted by Deepak Sharma <de...@gmail.com>.

I am interested in learning this fact as well as I need to put in
observability in a flink pipeline.

On Wed, 22 Sep 2021 at 8:40 PM, Dan Hill <qu...@gmail.com> wrote:

> Hi!
>
> I saw a recent Medium article
> <https://medium.com/pinterest-engineering/faster-flink-adoption-with-self-service-diagnosis-tool-at-pinterest-50a07143f444#id_token=eyJhbGciOiJSUzI1NiIsImtpZCI6ImMzMTA0YzY4OGMxNWU2YjhlNThlNjdhMzI4NzgwOTUyYjIxNzQwMTciLCJ0eXAiOiJKV1QifQ.eyJpc3MiOiJodHRwczovL2FjY291bnRzLmdvb2dsZS5jb20iLCJuYmYiOjE2MzIzMjI1OTUsImF1ZCI6IjIxNjI5NjAzNTgzNC1rMWs2cWUwNjBzMnRwMmEyamFtNGxqZGNtczAwc3R0Zy5hcHBzLmdvb2dsZXVzZXJjb250ZW50LmNvbSIsInN1YiI6IjExNzIzOTM4NzY4MTEyNzQ0NjE4OCIsImhkIjoicHJvbW90ZWQuYWkiLCJlbWFpbCI6ImRoaWxsQHByb21vdGVkLmFpIiwiZW1haWxfdmVyaWZpZWQiOnRydWUsImF6cCI6IjIxNjI5NjAzNTgzNC1rMWs2cWUwNjBzMnRwMmEyamFtNGxqZGNtczAwc3R0Zy5hcHBzLmdvb2dsZXVzZXJjb250ZW50LmNvbSIsIm5hbWUiOiJEYW4gSGlsbCIsInBpY3R1cmUiOiJodHRwczovL2xoMy5nb29nbGV1c2VyY29udGVudC5jb20vYS9BQVRYQUp4c3NCa2k0eklNNXJFXzVZZ0JoVHJ6NlAzVThkWWZoQVZIdkJrPXM5Ni1jIiwiZ2l2ZW5fbmFtZSI6IkRhbiIsImZhbWlseV9uYW1lIjoiSGlsbCIsImlhdCI6MTYzMjMyMjg5NSwiZXhwIjoxNjMyMzI2NDk1LCJqdGkiOiI3MjhjYTFkYjc5ZmVmMDg5NDliZGZiOGNmNTkyNzJhNjEyYWEwYjQyIn0.W-XUXxlly_L4S8OOqQdXd6hnJzWlaA2x_Iy9Uuvl3A-0wc-V-bEGbXxTb0gdhDZHCpzIs6U7wD04Mq1NXMv38FXEyf_BNkzPepby9lwT3CZxCG4XlQp7Eq7a5qrOmXcXZEjteRovJgOcxvJISAbnykXfQdyn0K4JOiT07EE_uOUd56o2V_OGFfxEcBJHMhkXedaICKbczz_-UowNhuwiIInAmhdqW9ZXo21_oBwWQCDKQL4vFgU-sWwxi2b-oa7bhzqNMP4dcWf0r8vvgxR4W4RqAI2oGvBAbE0-eCL8QvL0GPpi0t1tHl5Y57UG6PBvmP66cI2_OsWmN5GHTttB7g>
> by Pinterest on a diagnostic tool called DrSquirrel.  Looks great.
>
> What do other teams do for similar problems?
>
> E.g. Has anyone piped their Flink metadata into Apache Thirdeye?
>
> - Dan
>
-- 
Thanks
Deepak
www.bigdatabig.com
www.keosha.net