You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Gabor Somogyi (JIRA)" <ji...@apache.org> on 2019/07/12 12:03:00 UTC

[jira] [Created] (SPARK-28367) Kafka connector infinite wait because metadata never updated

Gabor Somogyi created SPARK-28367:
-------------------------------------

             Summary: Kafka connector infinite wait because metadata never updated
                 Key: SPARK-28367
                 URL: https://issues.apache.org/jira/browse/SPARK-28367
             Project: Spark
          Issue Type: Bug
          Components: Structured Streaming
    Affects Versions: 3.0.0
            Reporter: Gabor Somogyi


Spark uses an old and deprecated API named poll(long) which never returns and stays in live lock if metadata is not updated (for instance when broker disappears at consumer creation).

I've created a small standalone application to test it and the alternatives: https://github.com/gaborgsomogyi/kafka-get-assignment




--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org