You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@chukwa.apache.org by "Sourygna Luangsay (JIRA)" <ji...@apache.org> on 2012/11/01 20:27:12 UTC
[jira] [Created] (CHUKWA-668) Infinite loop in FileTailingAdaptor
Sourygna Luangsay created CHUKWA-668:
----------------------------------------
Summary: Infinite loop in FileTailingAdaptor
Key: CHUKWA-668
URL: https://issues.apache.org/jira/browse/CHUKWA-668
Project: Chukwa
Issue Type: Bug
Components: Data Collection
Affects Versions: 0.6.0
Reporter: Sourygna Luangsay
Fix For: 0.6.0
In LWFTAdaptor.java, lastSlurpTime is never updated, triggering an infinite loop if we use the
adaptor FileTailingAdaptor and subclass adaptors.
Loop occurs like this:
1) lastSlurpTime is never changed and thus always 0.
2) we append some data to the log file and we do a normal slurp
3) after this slurp, we have: len == fileReadOffset && fixedNameLastModified > lastSlurpTime
thus we think log file has rotated when it hasn't: fileReadOffset is reset to 0
4) we get: len > fileReadOffset and so we do a slurp of len bytes.
5) we are in the same situation as 3) -> infinite loop: the agent never stops sending chunks
to the collector although log files does not change.
It seems like this bug comes from CHUKWA-646. The first patch tailfile.patch seemed to do it
right but in the second patch, the line "lastSlurpTime = System.currentTimeMillis();" just
disapeared from LWFTAdaptor.java.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CHUKWA-668) Infinite loop in FileTailingAdaptor
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13493586#comment-13493586 ]
Eric Yang commented on CHUKWA-668:
----------------------------------
Hi Sourygna, could you write a test case to demo the problem? This will prevent future regressions. Thanks
> Infinite loop in FileTailingAdaptor
> -----------------------------------
>
> Key: CHUKWA-668
> URL: https://issues.apache.org/jira/browse/CHUKWA-668
> Project: Chukwa
> Issue Type: Bug
> Components: Data Collection
> Affects Versions: 0.6.0
> Reporter: Sourygna Luangsay
> Fix For: 0.6.0
>
> Attachments: chukwa-668.patch
>
>
> In LWFTAdaptor.java, lastSlurpTime is never updated, triggering an infinite loop if we use the
> adaptor FileTailingAdaptor and subclass adaptors.
> Loop occurs like this:
> 1) lastSlurpTime is never changed and thus always 0.
> 2) we append some data to the log file and we do a normal slurp
> 3) after this slurp, we have: len == fileReadOffset && fixedNameLastModified > lastSlurpTime
> thus we think log file has rotated when it hasn't: fileReadOffset is reset to 0
> 4) we get: len > fileReadOffset and so we do a slurp of len bytes.
> 5) we are in the same situation as 3) -> infinite loop: the agent never stops sending chunks
> to the collector although log files does not change.
> It seems like this bug comes from CHUKWA-646. The first patch tailfile.patch seemed to do it
> right but in the second patch, the line "lastSlurpTime = System.currentTimeMillis();" just
> disapeared from LWFTAdaptor.java.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CHUKWA-668) Infinite loop in FileTailingAdaptor
Posted by "Sourygna Luangsay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sourygna Luangsay updated CHUKWA-668:
-------------------------------------
Attachment: chukwa-668-2.patch
Added junit test in TestFileTailingAdaptors.java.
> Infinite loop in FileTailingAdaptor
> -----------------------------------
>
> Key: CHUKWA-668
> URL: https://issues.apache.org/jira/browse/CHUKWA-668
> Project: Chukwa
> Issue Type: Bug
> Components: Data Collection
> Affects Versions: 0.6.0
> Reporter: Sourygna Luangsay
> Fix For: 0.6.0
>
> Attachments: chukwa-668-2.patch, chukwa-668.patch
>
>
> In LWFTAdaptor.java, lastSlurpTime is never updated, triggering an infinite loop if we use the
> adaptor FileTailingAdaptor and subclass adaptors.
> Loop occurs like this:
> 1) lastSlurpTime is never changed and thus always 0.
> 2) we append some data to the log file and we do a normal slurp
> 3) after this slurp, we have: len == fileReadOffset && fixedNameLastModified > lastSlurpTime
> thus we think log file has rotated when it hasn't: fileReadOffset is reset to 0
> 4) we get: len > fileReadOffset and so we do a slurp of len bytes.
> 5) we are in the same situation as 3) -> infinite loop: the agent never stops sending chunks
> to the collector although log files does not change.
> It seems like this bug comes from CHUKWA-646. The first patch tailfile.patch seemed to do it
> right but in the second patch, the line "lastSlurpTime = System.currentTimeMillis();" just
> disapeared from LWFTAdaptor.java.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CHUKWA-668) Infinite loop in FileTailingAdaptor
Posted by "Sourygna Luangsay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sourygna Luangsay updated CHUKWA-668:
-------------------------------------
Attachment: chukwa-668.patch
> Infinite loop in FileTailingAdaptor
> -----------------------------------
>
> Key: CHUKWA-668
> URL: https://issues.apache.org/jira/browse/CHUKWA-668
> Project: Chukwa
> Issue Type: Bug
> Components: Data Collection
> Affects Versions: 0.6.0
> Reporter: Sourygna Luangsay
> Fix For: 0.6.0
>
> Attachments: chukwa-668.patch
>
>
> In LWFTAdaptor.java, lastSlurpTime is never updated, triggering an infinite loop if we use the
> adaptor FileTailingAdaptor and subclass adaptors.
> Loop occurs like this:
> 1) lastSlurpTime is never changed and thus always 0.
> 2) we append some data to the log file and we do a normal slurp
> 3) after this slurp, we have: len == fileReadOffset && fixedNameLastModified > lastSlurpTime
> thus we think log file has rotated when it hasn't: fileReadOffset is reset to 0
> 4) we get: len > fileReadOffset and so we do a slurp of len bytes.
> 5) we are in the same situation as 3) -> infinite loop: the agent never stops sending chunks
> to the collector although log files does not change.
> It seems like this bug comes from CHUKWA-646. The first patch tailfile.patch seemed to do it
> right but in the second patch, the line "lastSlurpTime = System.currentTimeMillis();" just
> disapeared from LWFTAdaptor.java.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CHUKWA-668) Infinite loop in FileTailingAdaptor
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501454#comment-13501454 ]
Hudson commented on CHUKWA-668:
-------------------------------
Integrated in Chukwa-trunk #460 (See [https://builds.apache.org/job/Chukwa-trunk/460/])
CHUKWA-668. Fix infinite loop in FileTailingAdaptor. (Sourygna Luangsay via Eric Yang) (Revision 1411818)
Result = SUCCESS
eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1411818
Files :
* /incubator/chukwa/trunk/CHANGES.txt
* /incubator/chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datacollection/adaptor/filetailer/FileTailingAdaptor.java
* /incubator/chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datacollection/adaptor/filetailer/LWFTAdaptor.java
* /incubator/chukwa/trunk/src/test/java/org/apache/hadoop/chukwa/datacollection/adaptor/filetailer/TestFileTailingAdaptors.java
> Infinite loop in FileTailingAdaptor
> -----------------------------------
>
> Key: CHUKWA-668
> URL: https://issues.apache.org/jira/browse/CHUKWA-668
> Project: Chukwa
> Issue Type: Bug
> Components: Data Collection
> Affects Versions: 0.6.0
> Reporter: Sourygna Luangsay
> Assignee: Sourygna Luangsay
> Fix For: 0.6.0
>
> Attachments: chukwa-668-2.patch, chukwa-668.patch
>
>
> In LWFTAdaptor.java, lastSlurpTime is never updated, triggering an infinite loop if we use the
> adaptor FileTailingAdaptor and subclass adaptors.
> Loop occurs like this:
> 1) lastSlurpTime is never changed and thus always 0.
> 2) we append some data to the log file and we do a normal slurp
> 3) after this slurp, we have: len == fileReadOffset && fixedNameLastModified > lastSlurpTime
> thus we think log file has rotated when it hasn't: fileReadOffset is reset to 0
> 4) we get: len > fileReadOffset and so we do a slurp of len bytes.
> 5) we are in the same situation as 3) -> infinite loop: the agent never stops sending chunks
> to the collector although log files does not change.
> It seems like this bug comes from CHUKWA-646. The first patch tailfile.patch seemed to do it
> right but in the second patch, the line "lastSlurpTime = System.currentTimeMillis();" just
> disapeared from LWFTAdaptor.java.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CHUKWA-668) Infinite loop in FileTailingAdaptor
Posted by "Sourygna Luangsay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13488963#comment-13488963 ]
Sourygna Luangsay commented on CHUKWA-668:
------------------------------------------
The patch I submit fix the loop problem.
It also covers 2 other small bugs:
- In LWFTAdaptor#tailFile(), return value hasMoreData was never updated, causing some problem
if the length of the data appended is greater than MAX_READ_SIZE_OPT.
- In FileTailingAdaptor#tailFile(), I think we shouldn't call slurp if len == fileReadOffset and
fixedNameLastModified <= lastSlurpTime (the file just stopped receiving more logs) so
I moved the slurp call in the "if (fixedNameLastModified > lastSlurpTime)" condition.
> Infinite loop in FileTailingAdaptor
> -----------------------------------
>
> Key: CHUKWA-668
> URL: https://issues.apache.org/jira/browse/CHUKWA-668
> Project: Chukwa
> Issue Type: Bug
> Components: Data Collection
> Affects Versions: 0.6.0
> Reporter: Sourygna Luangsay
> Fix For: 0.6.0
>
> Attachments: chukwa-668.patch
>
>
> In LWFTAdaptor.java, lastSlurpTime is never updated, triggering an infinite loop if we use the
> adaptor FileTailingAdaptor and subclass adaptors.
> Loop occurs like this:
> 1) lastSlurpTime is never changed and thus always 0.
> 2) we append some data to the log file and we do a normal slurp
> 3) after this slurp, we have: len == fileReadOffset && fixedNameLastModified > lastSlurpTime
> thus we think log file has rotated when it hasn't: fileReadOffset is reset to 0
> 4) we get: len > fileReadOffset and so we do a slurp of len bytes.
> 5) we are in the same situation as 3) -> infinite loop: the agent never stops sending chunks
> to the collector although log files does not change.
> It seems like this bug comes from CHUKWA-646. The first patch tailfile.patch seemed to do it
> right but in the second patch, the line "lastSlurpTime = System.currentTimeMillis();" just
> disapeared from LWFTAdaptor.java.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CHUKWA-668) Infinite loop in FileTailingAdaptor
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang updated CHUKWA-668:
-----------------------------
Status: Patch Available (was: Open)
+1 looks good.
> Infinite loop in FileTailingAdaptor
> -----------------------------------
>
> Key: CHUKWA-668
> URL: https://issues.apache.org/jira/browse/CHUKWA-668
> Project: Chukwa
> Issue Type: Bug
> Components: Data Collection
> Affects Versions: 0.6.0
> Reporter: Sourygna Luangsay
> Assignee: Sourygna Luangsay
> Fix For: 0.6.0
>
> Attachments: chukwa-668-2.patch, chukwa-668.patch
>
>
> In LWFTAdaptor.java, lastSlurpTime is never updated, triggering an infinite loop if we use the
> adaptor FileTailingAdaptor and subclass adaptors.
> Loop occurs like this:
> 1) lastSlurpTime is never changed and thus always 0.
> 2) we append some data to the log file and we do a normal slurp
> 3) after this slurp, we have: len == fileReadOffset && fixedNameLastModified > lastSlurpTime
> thus we think log file has rotated when it hasn't: fileReadOffset is reset to 0
> 4) we get: len > fileReadOffset and so we do a slurp of len bytes.
> 5) we are in the same situation as 3) -> infinite loop: the agent never stops sending chunks
> to the collector although log files does not change.
> It seems like this bug comes from CHUKWA-646. The first patch tailfile.patch seemed to do it
> right but in the second patch, the line "lastSlurpTime = System.currentTimeMillis();" just
> disapeared from LWFTAdaptor.java.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CHUKWA-668) Infinite loop in FileTailingAdaptor
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang reassigned CHUKWA-668:
--------------------------------
Assignee: Sourygna Luangsay
> Infinite loop in FileTailingAdaptor
> -----------------------------------
>
> Key: CHUKWA-668
> URL: https://issues.apache.org/jira/browse/CHUKWA-668
> Project: Chukwa
> Issue Type: Bug
> Components: Data Collection
> Affects Versions: 0.6.0
> Reporter: Sourygna Luangsay
> Assignee: Sourygna Luangsay
> Fix For: 0.6.0
>
> Attachments: chukwa-668-2.patch, chukwa-668.patch
>
>
> In LWFTAdaptor.java, lastSlurpTime is never updated, triggering an infinite loop if we use the
> adaptor FileTailingAdaptor and subclass adaptors.
> Loop occurs like this:
> 1) lastSlurpTime is never changed and thus always 0.
> 2) we append some data to the log file and we do a normal slurp
> 3) after this slurp, we have: len == fileReadOffset && fixedNameLastModified > lastSlurpTime
> thus we think log file has rotated when it hasn't: fileReadOffset is reset to 0
> 4) we get: len > fileReadOffset and so we do a slurp of len bytes.
> 5) we are in the same situation as 3) -> infinite loop: the agent never stops sending chunks
> to the collector although log files does not change.
> It seems like this bug comes from CHUKWA-646. The first patch tailfile.patch seemed to do it
> right but in the second patch, the line "lastSlurpTime = System.currentTimeMillis();" just
> disapeared from LWFTAdaptor.java.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira