Apache Flink - Rest API for num of records in/out

[M Singh <ma...@yahoo.com>]

Hi Folks:
I am trying to find if I can get the number of records for an operator using flinks REST API.  I've checked the docs at https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/ops/rest_api/.   
I did see some apis that use vertexid, but could not find how to that info without having vertex ids.
I am using flink 1.14.4. 
Can you please let me know how to get that ?
Thanks

Re: Apache Flink - Rest API for num of records in/out

[M Singh <ma...@yahoo.com>]

 Hi Andreas:
Thanks for your pointers and reference.
I will check it out.

    On Tuesday, June 7, 2022, 05:17:43 PM EDT, Hailu, Andreas <an...@gs.com> wrote:  
 
 <!--#yiv4450489405 _filtered {} _filtered {} _filtered {}#yiv4450489405 #yiv4450489405 p.yiv4450489405MsoNormal, #yiv4450489405 li.yiv4450489405MsoNormal, #yiv4450489405 div.yiv4450489405MsoNormal {margin:0in;margin-bottom:.0001pt;font-size:12.0pt;font-family:"Times New Roman", serif;}#yiv4450489405 a:link, #yiv4450489405 span.yiv4450489405MsoHyperlink {color:blue;text-decoration:underline;}#yiv4450489405 a:visited, #yiv4450489405 span.yiv4450489405MsoHyperlinkFollowed {color:purple;text-decoration:underline;}#yiv4450489405 p.yiv4450489405MsoListParagraph, #yiv4450489405 li.yiv4450489405MsoListParagraph, #yiv4450489405 div.yiv4450489405MsoListParagraph {margin-top:0in;margin-right:0in;margin-bottom:0in;margin-left:.5in;margin-bottom:.0001pt;font-size:12.0pt;font-family:"Times New Roman", serif;}#yiv4450489405 p.yiv4450489405msonormal0, #yiv4450489405 li.yiv4450489405msonormal0, #yiv4450489405 div.yiv4450489405msonormal0 {margin-right:0in;margin-left:0in;font-size:12.0pt;font-family:"Times New Roman", serif;}#yiv4450489405 span.yiv4450489405EmailStyle18 {font-family:"Calibri", sans-serif;color:#1F497D;}#yiv4450489405 .yiv4450489405MsoChpDefault {font-size:10.0pt;} _filtered {}#yiv4450489405 div.yiv4450489405WordSection1 {}#yiv4450489405 _filtered {} _filtered {} _filtered {} _filtered {} _filtered {} _filtered {} _filtered {} _filtered {} _filtered {} _filtered {}#yiv4450489405 ol {margin-bottom:0in;}#yiv4450489405 ul {margin-bottom:0in;}-->
Hi M,
 
  
 
We had a similar requirement – we were able to solve for this by:
 
1.      Supply the operators we’re interested in acquiring metrics for through the various name() methods
 
2.      Use the jobid API [1] and find the operator we’ve named in the “vertices” array
 
  
 
[1]https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/ops/rest_api/#jobs-jobid
 
  
 
ah
 
  
 
From: M Singh <ma...@yahoo.com>
Sent: Tuesday, June 7, 2022 4:51 PM
To: User-Flink <us...@flink.apache.org>
Subject: Apache Flink - Rest API for num of records in/out
 
  
 
Hi Folks:
 
  
 
I am trying to find if I can get the number of records for an operator using flinks REST API.  I've checked the docs at https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/ops/rest_api/.   
 
  
 
I did see some apis that use vertexid, but could not find how to that info without having vertex ids.
 
  
 
I am using flink 1.14.4. 
 
  
 
Can you please let me know how to get that ?
 
  
 
Thanks
 

Your Personal Data: We may collect and process information about you that may be subject to data protection laws. For more information about how we use and disclose your personal data, how we protect your information, our legal basis to use your information, your rights and who you can contact, please refer to: www.gs.com/privacy-notices
  

RE: Apache Flink - Rest API for num of records in/out

["Hailu, Andreas" <An...@gs.com>]

Hi M,

We had a similar requirement – we were able to solve for this by:

1.       Supply the operators we’re interested in acquiring metrics for through the various name() methods

2.       Use the jobid API [1] and find the operator we’ve named in the “vertices” array

[1] https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/ops/rest_api/#jobs-jobid

ah

From: M Singh <ma...@yahoo.com>
Sent: Tuesday, June 7, 2022 4:51 PM
To: User-Flink <us...@flink.apache.org>
Subject: Apache Flink - Rest API for num of records in/out

Hi Folks:

I am trying to find if I can get the number of records for an operator using flinks REST API.  I've checked the docs at https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/ops/rest_api/<https://urldefense.proofpoint.com/v2/url?u=https-3A__nightlies.apache.org_flink_flink-2Ddocs-2Drelease-2D1.15_docs_ops_rest-5Fapi_&d=DwMFaQ&c=7563p3e2zaQw0AB1wrFVgyagb2IE5rTZOYPxLxfZlX4&r=hRr4SA7BtUvKoMBP6VDhfisy2OJ1ZAzai-pcCC6TFXM&m=o3eE3zbNzc9JgqOXl2UCi8v6yX8FdAUjaCvG4y4U9spp4S6FfQ_G9PeKZcTpvX9M&s=2HVrYEfguyQunm6-L98slYnMrzekiYxuBvU1vzhI9b8&e=>.

I did see some apis that use vertexid, but could not find how to that info without having vertex ids.

I am using flink 1.14.4.

Can you please let me know how to get that ?

Thanks

________________________________

Your Personal Data: We may collect and process information about you that may be subject to data protection laws. For more information about how we use and disclose your personal data, how we protect your information, our legal basis to use your information, your rights and who you can contact, please refer to: www.gs.com/privacy-notices<http://www.gs.com/privacy-notices>