Metrics outside of RichFunction context

[John Karp <jo...@verizonmedia.com>]

Hi,

I'm using StreamingFileSink to collect records into avro files. Inside of
the BulkWriter implementation, I have to do some operations (such as
dynamic schema lookup) which I want to record metrics about. However, the
BulkWriter API, as it is defined, does not accept a RuntimeContext or
MetricsGroup from the StreamingFileSink, so I seemingly don't have access
to the metrics API. And it doesn't look like StreamingFileSink is
particularly extensible. What alternatives do I have besides forking
StreamingFileSink, or bypassing the Flink metrics API entirely?

Thanks,
John

Re: Metrics outside of RichFunction context

[Chesnay Schepler <ch...@apache.org>]

I don't believe there are other options, outside of creating a ticket to 
have Flink extend the API accordingly.

On 19/08/2021 16:40, John Karp wrote:
> Hi,
>
> I'm using StreamingFileSink to collect records into avro files. Inside 
> of the BulkWriter implementation, I have to do some operations (such 
> as dynamic schema lookup) which I want to record metrics about. 
> However, the BulkWriter API, as it is defined, does not accept a 
> RuntimeContext or MetricsGroup from the StreamingFileSink, so I 
> seemingly don't have access to the metrics API. And it doesn't look 
> like StreamingFileSink is particularly extensible. What alternatives 
> do I have besides forking StreamingFileSink, or bypassing the Flink 
> metrics API entirely?
>
> Thanks,
> John