[jira] [Resolved] (SPARK-1916) SparkFlumeEvent with body bigger than 1020 bytes are not read properly

["Patrick Wendell (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-1916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Patrick Wendell resolved SPARK-1916.
------------------------------------

       Resolution: Fixed
    Fix Version/s: 0.9.2
                   1.0.1

Issue resolved by pull request 865
[https://github.com/apache/spark/pull/865]

> SparkFlumeEvent with body bigger than 1020 bytes are not read properly
> ----------------------------------------------------------------------
>
>                 Key: SPARK-1916
>                 URL: https://issues.apache.org/jira/browse/SPARK-1916
>             Project: Spark
>          Issue Type: Bug
>          Components: Streaming
>    Affects Versions: 0.9.0
>            Reporter: David Lemieux
>            Assignee: David Lemieux
>             Fix For: 1.0.1, 0.9.2
>
>         Attachments: SPARK-1916.diff
>
>
> The readExternal implementation on SparkFlumeEvent will read only the first 1020 bytes of the actual body when streaming data from flume.
> This means that any event sent to Spark via Flume will be processed properly if the body is small, but will fail if the body is bigger than 1020.
> Considering that the default max size for a Flume Avro Event is 32K, the implementation should be updated to read more.
> The following is related : http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-using-Flume-body-size-limitation-tt6127.html



--
This message was sent by Atlassian JIRA
(v6.2#6252)