You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Felipe Gutierrez <fe...@gmail.com> on 2020/03/16 16:17:13 UTC
How do I get the outPoolUsage value inside my own stream operator?
Hi community,
I have built my own operator (not a UDF) and I want to collect the
metrics of "outPoolUsage" inside it. How do I do it assuming that I
have to do some modifications in the source code?
I know that the Gouge comes from
flink-runtime/org.apache.flink.runtime.io.network.metrics.OutputBufferPoolUsageGauge.java.
Inside of my operator MyAbstractUdfStreamOperator I can get the
"MetricGroup metricGroup = this.getMetricGroup()".
Then I implemented the "Gauge<Float> gauge = (Gauge<Float>)
metricGroup.getMetric("outPoolUsage");" but it returns null all the
time. Even when I click on the Backpressure UI Interface.
Thanks,
Felipe
--
-- Felipe Gutierrez
-- skype: felipe.o.gutierrez
-- https://felipeogutierrez.blogspot.com
Re: How do I get the outPoolUsage value inside my own stream operator?
Posted by Zhijiang <wa...@aliyun.com>.
Hi Felipe,
I checked the code path, and the metric of outPoolUsage is under the following layer: TaskMetricGroup -> TaskIOMetricGroup -> "buffers" group -> "outPoolUsage".
It seems that you missed the `TaskIOMetricGroup` from below samples. You can get it from TaskMetricGroup.
Hope it solve your problem.
Best,
Zhijiang
------------------------------------------------------------------
From:Felipe Gutierrez <fe...@gmail.com>
Send Time:2020 Mar. 17 (Tue.) 17:50
To:user <us...@flink.apache.org>
Subject:Re: How do I get the outPoolUsage value inside my own stream operator?
Hi,
just for the record. I am collecting the values of outPoolUsage using
the piece of code below inside my stream operator
OperatorMetricGroup operatorMetricGroup = (OperatorMetricGroup)
this.getMetricGroup();
TaskMetricGroup taskMetricGroup = operatorMetricGroup.parent();
MetricGroup metricGroup = taskMetricGroup.getGroup("buffers");
Gauge<Float> gauge = (Gauge<Float>) metricGroup.getMetric("outPoolUsage");
if (gauge != null && gauge.getValue() != null) {
float outPoolUsage = gauge.getValue().floatValue();
this.outPoolUsageHistogram.update((long) (outPoolUsage * 100));
}
--
-- Felipe Gutierrez
-- skype: felipe.o.gutierrez
-- https://felipeogutierrez.blogspot.com
On Mon, Mar 16, 2020 at 5:17 PM Felipe Gutierrez
<fe...@gmail.com> wrote:
>
> Hi community,
>
> I have built my own operator (not a UDF) and I want to collect the
> metrics of "outPoolUsage" inside it. How do I do it assuming that I
> have to do some modifications in the source code?
>
> I know that the Gouge comes from
> flink-runtime/org.apache.flink.runtime.io.network.metrics.OutputBufferPoolUsageGauge.java.
> Inside of my operator MyAbstractUdfStreamOperator I can get the
> "MetricGroup metricGroup = this.getMetricGroup()".
> Then I implemented the "Gauge<Float> gauge = (Gauge<Float>)
> metricGroup.getMetric("outPoolUsage");" but it returns null all the
> time. Even when I click on the Backpressure UI Interface.
>
> Thanks,
> Felipe
> --
> -- Felipe Gutierrez
> -- skype: felipe.o.gutierrez
> -- https://felipeogutierrez.blogspot.com
Re: How do I get the outPoolUsage value inside my own stream operator?
Posted by Felipe Gutierrez <fe...@gmail.com>.
Hi,
just for the record. I am collecting the values of outPoolUsage using
the piece of code below inside my stream operator
OperatorMetricGroup operatorMetricGroup = (OperatorMetricGroup)
this.getMetricGroup();
TaskMetricGroup taskMetricGroup = operatorMetricGroup.parent();
MetricGroup metricGroup = taskMetricGroup.getGroup("buffers");
Gauge<Float> gauge = (Gauge<Float>) metricGroup.getMetric("outPoolUsage");
if (gauge != null && gauge.getValue() != null) {
float outPoolUsage = gauge.getValue().floatValue();
this.outPoolUsageHistogram.update((long) (outPoolUsage * 100));
}
--
-- Felipe Gutierrez
-- skype: felipe.o.gutierrez
-- https://felipeogutierrez.blogspot.com
On Mon, Mar 16, 2020 at 5:17 PM Felipe Gutierrez
<fe...@gmail.com> wrote:
>
> Hi community,
>
> I have built my own operator (not a UDF) and I want to collect the
> metrics of "outPoolUsage" inside it. How do I do it assuming that I
> have to do some modifications in the source code?
>
> I know that the Gouge comes from
> flink-runtime/org.apache.flink.runtime.io.network.metrics.OutputBufferPoolUsageGauge.java.
> Inside of my operator MyAbstractUdfStreamOperator I can get the
> "MetricGroup metricGroup = this.getMetricGroup()".
> Then I implemented the "Gauge<Float> gauge = (Gauge<Float>)
> metricGroup.getMetric("outPoolUsage");" but it returns null all the
> time. Even when I click on the Backpressure UI Interface.
>
> Thanks,
> Felipe
> --
> -- Felipe Gutierrez
> -- skype: felipe.o.gutierrez
> -- https://felipeogutierrez.blogspot.com