You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Julio Biason <ju...@azion.com> on 2018/04/24 17:59:07 UTC
Testing Metrics
Hey guys and gals,
Just wondering: Does anyone have an idea how to test if metrics are being
generated? I have an integration test and I just added a processor to count
elements in late arrivals (the general idea is to capture those, count and
get an average, so we can adjust the allowedLateness) but now I'm wondering
if there is a way I can integrate this into the test itself.
Possible? Not possible? Ideas?
--
*Julio Biason*, Sofware Engineer
*AZION* | Deliver. Accelerate. Protect.
Office: +55 51 3083 8101 <callto:+555130838101> | Mobile: +55 51
<callto:+5551996209291>*99907 0554*
Re: Testing Metrics
Posted by Chesnay Schepler <ch...@apache.org>.
+1 to using reporters.
You will have to explicitly pass a configuration with the reporter
settings to the environment via
StreamExecutionEnvironment#createLocalEnvironment(int, Configuration).
The reporter can verify registrations/values and pass this information
back to the main test thread through a static field (for simplicity).
On 25.04.2018 09:11, Tzu-Li (Gordon) Tai wrote:
> Hi,
>
> Do you mean tests to verify that some metric is actually registered?
>
> AFAIK, this is not really easy to do as a unit test.
>
> One possible way is to have an integration test that uses a metrics
> reporter, from which you verify against.
> For example, the Kafka consumer integration tests that uses the
> JMXReporter for a submitted job and queries results from that to
> verify that some specific metrics are forwarded from the Kafka client.
> We also have an end-to-end test for savepointing, that uses the
> SLF4JReporter to write metrics to logs, which are parsed for a
> specific metric to monitor progress.
>
> Also pulling in Chesnay (in CC), who might have more ideas here.
>
> Cheers,
> Gordon
>
> On 25 April 2018 at 1:59:18 AM, Julio Biason (julio.biason@azion.com
> <ma...@azion.com>) wrote:
>
>> Hey guys and gals,
>>
>> Just wondering: Does anyone have an idea how to test if metrics are
>> being generated? I have an integration test and I just added a
>> processor to count elements in late arrivals (the general idea is to
>> capture those, count and get an average, so we can adjust the
>> allowedLateness) but now I'm wondering if there is a way I can
>> integrate this into the test itself.
>>
>> Possible? Not possible? Ideas?
>>
>> --
>> *Julio Biason*, Sofware Engineer
>> *AZION*| Deliver. Accelerate. Protect.
>> Office: +55 51 3083 8101 <callto:+555130838101> | Mobile: +55 51
>> <callto:+5551996209291>_99907 0554_
Re: Testing Metrics
Posted by "Tzu-Li (Gordon) Tai" <tz...@apache.org>.
Hi,
Do you mean tests to verify that some metric is actually registered?
AFAIK, this is not really easy to do as a unit test.
One possible way is to have an integration test that uses a metrics reporter, from which you verify against.
For example, the Kafka consumer integration tests that uses the JMXReporter for a submitted job and queries results from that to verify that some specific metrics are forwarded from the Kafka client.
We also have an end-to-end test for savepointing, that uses the SLF4JReporter to write metrics to logs, which are parsed for a specific metric to monitor progress.
Also pulling in Chesnay (in CC), who might have more ideas here.
Cheers,
Gordon
On 25 April 2018 at 1:59:18 AM, Julio Biason (julio.biason@azion.com) wrote:
Hey guys and gals,
Just wondering: Does anyone have an idea how to test if metrics are being generated? I have an integration test and I just added a processor to count elements in late arrivals (the general idea is to capture those, count and get an average, so we can adjust the allowedLateness) but now I'm wondering if there is a way I can integrate this into the test itself.
Possible? Not possible? Ideas?
--
Julio Biason, Sofware Engineer
AZION | Deliver. Accelerate. Protect.
Office: +55 51 3083 8101 | Mobile: +55 51 99907 0554