Web Dashboard reports 0 for sent/received bytes and records

[Mohammad Kargar <mk...@phemi.com>]

Hi,

I have a simple job where FlinkKafkaConsumer08 reads data from a topic
(with 8 partitions) and a custom Sink implementation inserts the data into
Accumulo database. I can verify that the data is getting inserted into the
db by adding simple system.out.println(...) to the invoke method of the
sink and also monitoring Accumulo stats. However Flink Web Dashboard
reports 0 for sent/received bytes and records (see attached file for a
snapshot). Any thoughts?

*Hadoop platform*: Hortonworks Data Platform 2.5
*Flink*: flink-1.2.0-bin-hadoop27-scala_2.10

Thanks,
Mohammad

Re: Web Dashboard reports 0 for sent/received bytes and records

[Konstantin Gregor <ko...@tngtech.com>]

Hi Mohammad,

that's normal. The source will never have a value other than zero in the
"Records received" column, only in the "Records sent" column. But since
your whole program is chained to one subtask, the source will never send
anything to another subtask, because there is none.
Same for the sink, it will only count values it receives from its
preceding subtask, but due to chaining, there is no preceding subtask.
Thus there will only be zeros in your dashboard.
You can add your own Accumulators or Metrics and see the number of
records in your operators through that, or you could also disable the
chaining (but chaining is a good thing for performance, so you probably
don't want to do that).

Hope that helps,
best regards

Konstantin

On 30.03.2017 16:48, Mohammad Kargar wrote:
> Hi,
> 
> I have a simple job where FlinkKafkaConsumer08 reads data from a topic
> (with 8 partitions) and a custom Sink implementation inserts the data
> into Accumulo database. I can verify that the data is getting inserted
> into the db by adding simple system.out.println(...) to the invoke
> method of the sink and also monitoring Accumulo stats. However Flink Web
> Dashboard reports 0 for sent/received bytes and records (see attached
> file for a snapshot). Any thoughts?
> 
> *Hadoop platform*: Hortonworks Data Platform 2.5
> *Flink*: flink-1.2.0-bin-hadoop27-scala_2.10
> 
> Thanks,
> Mohammad

-- 
Konstantin Gregor * konstantin.gregor@tngtech.com
TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterföhring
Geschäftsführer: Henrik Klagges, Christoph Stock, Dr. Robert Dahlke
Sitz: Unterföhring * Amtsgericht München * HRB 135082