You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by "Sendoh (JIRA)" <ji...@apache.org> on 2016/12/19 15:08:58 UTC

[jira] [Created] (FLINK-5368) Let Kafka consumer show something when it fails to read one topic out of topic list

Sendoh created FLINK-5368:
-----------------------------

             Summary: Let Kafka consumer show something when it fails to read one topic out of topic list
                 Key: FLINK-5368
                 URL: https://issues.apache.org/jira/browse/FLINK-5368
             Project: Flink
          Issue Type: Improvement
          Components: Kafka Connector
            Reporter: Sendoh
            Assignee: Sendoh
            Priority: Minor


As a developer when reading data from many topics, I want Kafka consumer to throw Exception or LOG.error() if any topic is not available. The motivation is we read many topics as list at one time, and sometimes we fail to recognize that one or two topics' names have been changed or deprecated, and Flink Kafka connector doesn't show the error.

My proposed change would be either to throw RunTimeError or to use log.Error(topic + "doesn't have any partition") if partitionsForTopic is null at this function. 
https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka-0.9/src/main/java/org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer09.java#L208

Any suggestion is welcome.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [jira] [Created] (FLINK-5368) Let Kafka consumer show something when it fails to read one topic out of topic list

Posted by Jark Wu <wu...@alibaba-inc.com>.

Hi Sednoh,

It seems that the Travis error has nothing to do with the code you changed. 


- Jark Wu 

> 在 2016年12月21日，下午7:32，Hung Chang <un...@gmail.com> 写道：
> 
> Hi Flink developers,
> 
> Travis CI shows error when I send PR
> https://github.com/apache/flink/pull/3036, but I don't find the issue
> reading the log.
> Does anyone know what I should do to fix it?
> 
> test:
> https://travis-ci.org/apache/flink/jobs/185728861
> 
> full log:
> https://s3.amazonaws.com/archive.travis-ci.org/jobs/185728861/log.txt
> 
> 
> Best,
> 
> Sednoh
> 
> 
> 
> 
> --
> View this message in context: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/jira-Created-FLINK-5368-Let-Kafka-consumer-show-something-when-it-fails-to-read-one-topic-out-of-topt-tp15047p15079.html
> Sent from the Apache Flink Mailing List archive. mailing list archive at Nabble.com.

Re: [jira] [Created] (FLINK-5368) Let Kafka consumer show something when it fails to read one topic out of topic list

Posted by Hung Chang <un...@gmail.com>.

Hi Flink developers,

Travis CI shows error when I send PR
https://github.com/apache/flink/pull/3036, but I don't find the issue
reading the log.
Does anyone know what I should do to fix it?

test:
https://travis-ci.org/apache/flink/jobs/185728861

full log:
https://s3.amazonaws.com/archive.travis-ci.org/jobs/185728861/log.txt


Best,

Sednoh




--
View this message in context: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/jira-Created-FLINK-5368-Let-Kafka-consumer-show-something-when-it-fails-to-read-one-topic-out-of-topt-tp15047p15079.html
Sent from the Apache Flink Mailing List archive. mailing list archive at Nabble.com.