You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@bahir.apache.org by "Prashant Sharma (JIRA)" <ji...@apache.org> on 2018/04/27 12:14:00 UTC

[jira] [Commented] (BAHIR-164) Spark Streaming with MQTT fails with Spark 2.3.0

    [ https://issues.apache.org/jira/browse/BAHIR-164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16456281#comment-16456281 ] 

Prashant Sharma commented on BAHIR-164:
---------------------------------------

MQTT source has to be migrated to new DataSourceV2 API.

> Spark Streaming with MQTT fails with Spark 2.3.0
> ------------------------------------------------
>
>                 Key: BAHIR-164
>                 URL: https://issues.apache.org/jira/browse/BAHIR-164
>             Project: Bahir
>          Issue Type: Bug
>          Components: Spark Streaming Connectors
>    Affects Versions: Spark-2.2.0, Spark-2.3
>            Reporter: Jussi-Pekka Partanen
>            Priority: Blocker
>
> Currently I’m not able to use latest Spark 2.3.0 release together with Bahir, but getting the following error:
>  
> _java.lang.AssertionError: assertion failed: DataFrame returned by getBatch from org.apache.bahir.sql.streaming.mqtt.MQTTTextStreamSource@66847d45 did not have isStreaming=true_
>  
> org/apache/bahir/sql/streaming/mqtt/MQTTStreamSource.scala line 164 seems to be causing the issue, i.e. it’s returning a normal dataframe instead of streaming dataframe.
>  
> There seems to be discussion about the same topic, but on stream source for Kafka: https://lists.apache.org/thread.html/015f2fbe9fbeafb86214a969a2d883a4898621189a86f21a4c6fbb75@%3Cdev.spark.apache.org%3E



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)