You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by "PoolakkalMukkath, Shakir" <Sh...@comcast.com> on 2019/05/15 15:01:09 UTC

Re: User Interface not showing the actual count received and produced

Hi Flink team,



I am developing a flow which uses

  *   FlinkKafkaConsumer010 to consume message from Kafka  and
  *   FlinkKinesisProducer to produce the results to Kinesis.



In the user interface, I always see Bytes and Record received from Kafka is zero even though it is receiving events and processing.  And same with Kinesis Sink, Bytes and record sent is always zero even though it is posting events to Kinesis.



Any reason why my UI is not showing the actual count ? Thanks





Thanks,

Shakir

Re: User Interface not showing the actual count received and produced

Posted by Fabian Hueske <fh...@gmail.com>.

Hi Shakir,

This is a frequently reported issue in Flink's metrics collection / UI.
Send and received records and bytes only include data that is shared
between Flink tasks but not between a source system (Kafka) and Flink or
Flink and a sink system (Kinesis).
IIRC, there is an effort to fix this problem.

Best, Fabian

Am Mi., 15. Mai 2019 um 17:04 Uhr schrieb PoolakkalMukkath, Shakir <
Shakir_PoolakkalMukkath@comcast.com>:

>
>
> Hi Flink team,
>
>
>
> I am developing a flow which uses
>
>    - FlinkKafkaConsumer010 to consume message from Kafka  and
>    - FlinkKinesisProducer to produce the results to Kinesis.
>
>
>
> In the user interface, I always see Bytes and Record received from Kafka
> is zero even though it is receiving events and processing.  And same with
> Kinesis Sink, Bytes and record sent is always zero even though it is
> posting events to Kinesis.
>
>
>
> Any reason why my UI is not showing the actual count ? Thanks
>
>
>
>
>
> Thanks,
>
> Shakir
>

Re: User Interface not showing the actual count received and produced

Posted by "PoolakkalMukkath, Shakir" <Sh...@comcast.com>.


Hi Flink team,



I am developing a flow which uses

  *   FlinkKafkaConsumer010 to consume message from Kafka  and
  *   FlinkKinesisProducer to produce the results to Kinesis.



In the user interface, I always see Bytes and Record received from Kafka is zero even though it is receiving events and processing.  And same with Kinesis Sink, Bytes and record sent is always zero even though it is posting events to Kinesis.



Any reason why my UI is not showing the actual count ? Thanks





Thanks,

Shakir