[jira] [Created] (FLUME-2360) Spooling Source: LineDeserializer incorrectly stops reading events if UTF-8 char occurs at end of buffer

["Keith Wright (JIRA)" <ji...@apache.org>]

Keith Wright created FLUME-2360:
-----------------------------------

             Summary: Spooling Source: LineDeserializer incorrectly stops reading events if UTF-8 char occurs at end of buffer
                 Key: FLUME-2360
                 URL: https://issues.apache.org/jira/browse/FLUME-2360
             Project: Flume
          Issue Type: Bug
          Components: Sinks+Sources
    Affects Versions: v1.4.0
         Environment: CDH 4.4
            Reporter: Keith Wright


We are using a Spooling source to read from files and have noticed that there are situations where the source will stop reading in the middle of a file (see attached example file).  After much debugging, I have determine that this occurs if the buffer in ResettableFileInputStream happens to fill when in the middle of handling a UTF-8 character.  In this case readChar() will return -1 which ends the read lines loop in LineSerializer.  Running the spool source with the attached file should result in > 5400 events in the channel but now it will only return ~3300.



--
This message was sent by Atlassian JIRA
(v6.2#6252)