[jira] [Assigned] (FLUME-533) TailSource line length limitation

["Jonathan Hsieh (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLUME-533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jonathan Hsieh reassigned FLUME-533:
------------------------------------

    Assignee: Jonathan Hsieh  (was: Disabled imported user)

> TailSource line length limitation
> ---------------------------------
>
>                 Key: FLUME-533
>                 URL: https://issues.apache.org/jira/browse/FLUME-533
>             Project: Flume
>          Issue Type: Bug
>          Components: Sinks+Sources
>    Affects Versions: v0.9.3
>            Reporter: Disabled imported user
>            Assignee: Jonathan Hsieh
>              Labels: TailSource, line_length, sink
>             Fix For: v0.9.5
>
>         Attachments: 0001-Fix-line-length-limitation-in-TailSource.patch
>
>
> It appears that having a line with length of 32767 bytes or more
> throws the TailSource into a, um, tail spin.  I've set
> flume.event.max.size.bytes to 1000000, so that shouldn't be the
> limitation.  Simple steps to reproduce:
> flume-0.9.3rc1> ruby -e 'puts "x"*32767' > /tmp/log
> flume-0.9.3rc1> bin/flume dump 'tail("/tmp/log")'
> It appears that the source sends an incomplete message to the sink,
> and then continuously retries to send it. 
> I've attached a patch that creates a buffer of size flume.event.max.size.bytes, 
> and if it detects a line longer than this, it gets thrown away.  It's been working
> for our systems for a week, but I haven't had time to write a unit test and test
> the edge cases.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira