You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Ian Brooks (JIRA)" <ji...@apache.org> on 2016/07/20 15:08:20 UTC
[jira] [Commented] (SPARK-3210) Flume Polling Receiver must be more
tolerant to connection failures.
[ https://issues.apache.org/jira/browse/SPARK-3210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386002#comment-15386002 ]
Ian Brooks commented on SPARK-3210:
-----------------------------------
Hi,
I have also noticed an issue with resiliency for the flume polling receiver. This issue I have is as follows
1. Start Flume agent then Spark application
2. Spark application correctly connects to the flume agent and can receive data what is sent to Flume
3. Restart Flume
4. Spark application doesn't detect that Flume has been restarted and as such doesn't reconnect at any point preventing the Spark application from receiving any more data until its restarted.
I've had a trawl through the documentation and source code for FlumeUtils.createPollingStream but couldn't see anyway to test and reconnect if needed.
-Ian
> Flume Polling Receiver must be more tolerant to connection failures.
> --------------------------------------------------------------------
>
> Key: SPARK-3210
> URL: https://issues.apache.org/jira/browse/SPARK-3210
> Project: Spark
> Issue Type: Bug
> Components: Streaming
> Reporter: Hari Shreedharan
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org