You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Kai Fu <zz...@gmail.com> on 2021/06/04 01:31:33 UTC

Flink exported metrics scope configuration

Hi team,

We noticed that Prometheus metrics exporter exports all of the metrics at
the most fine-grained level, which is tremendous for the prometheus server
especially when the parallelism is high. The metrics volume crawled from a
single host(parallelism 8) is around 40MB for us currently. This is
due to *task_name
*attribute in the metrics generated by the engine being very long. The
task_name attribute is auto-generated from SQL job, and it seems it's
attaching all field names onto it.

We want to reduce the metrics volume by either drop task_name or at some
more coarse-grained level. But I cannot find any related documents about
this, any advice on that?

[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/deployment/metric_reporters/
[2]
https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/ops/metrics/#list-of-all-variables

-- 
*Best wishes,*
*- Kai*

Re: Flink exported metrics scope configuration

Posted by Kai Fu <zz...@gmail.com>.
Hi Mason,

Thank you for the advice, as I tried, it works and reduces a lot in size.

On Fri, Jun 4, 2021 at 11:45 AM Mason Chen <ma...@apple.com> wrote:

> Hi Kai,
>
> You can use the excluded variables config for the reporter.
>
>    - metrics.reporter.<name>.scope.variables.excludes: (optional) A
>    semi-colon (;) separate list of variables that should be ignored by
>    tag-based reporters (e.g., Prometheus, InfluxDB).
>
>
>
> https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/deployment/metric_reporters/#reporter
>
> Best,
> Mason
>
> On Jun 3, 2021, at 9:31 PM, Kai Fu <zz...@gmail.com> wrote:
>
> Hi team,
>
> We noticed that Prometheus metrics exporter exports all of the metrics at
> the most fine-grained level, which is tremendous for the prometheus server
> especially when the parallelism is high. The metrics volume crawled from a
> single host(parallelism 8) is around 40MB for us currently. This is due to *task_name
> *attribute in the metrics generated by the engine being very long. The
> task_name attribute is auto-generated from SQL job, and it seems it's
> attaching all field names onto it.
>
> We want to reduce the metrics volume by either drop task_name or at some
> more coarse-grained level. But I cannot find any related documents about
> this, any advice on that?
>
> [1]
> https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/deployment/metric_reporters/
> [2]
> https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/ops/metrics/#list-of-all-variables
>
> --
> *Best wishes,*
> *- Kai*
>
>
>

-- 
*Best wishes,*
*- Kai*

Re: Flink exported metrics scope configuration

Posted by Mason Chen <ma...@apple.com>.
Hi Kai,

You can use the excluded variables config for the reporter. 
metrics.reporter.<name>.scope.variables.excludes: (optional) A semi-colon (;) separate list of variables that should be ignored by tag-based reporters (e.g., Prometheus, InfluxDB).

https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/deployment/metric_reporters/#reporter <https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/deployment/metric_reporters/#reporter>

Best,
Mason

> On Jun 3, 2021, at 9:31 PM, Kai Fu <zz...@gmail.com> wrote:
> 
> Hi team,
> 
> We noticed that Prometheus metrics exporter exports all of the metrics at the most fine-grained level, which is tremendous for the prometheus server especially when the parallelism is high. The metrics volume crawled from a single host(parallelism 8) is around 40MB for us currently. This is due to task_name attribute in the metrics generated by the engine being very long. The task_name attribute is auto-generated from SQL job, and it seems it's attaching all field names onto it.
> 
> We want to reduce the metrics volume by either drop task_name or at some more coarse-grained level. But I cannot find any related documents about this, any advice on that? 
> 
> [1] https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/deployment/metric_reporters/ <https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/deployment/metric_reporters/>
> [2] https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/ops/metrics/#list-of-all-variables <https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/ops/metrics/#list-of-all-variables>
> 
> -- 
> Best wishes,
> - Kai