You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Joel Koshy (JIRA)" <ji...@apache.org> on 2012/06/30 00:30:44 UTC
[jira] [Created] (KAFKA-384) Fix intermittent test failures and
remove unnecessary sleeps
Joel Koshy created KAFKA-384:
--------------------------------
Summary: Fix intermittent test failures and remove unnecessary sleeps
Key: KAFKA-384
URL: https://issues.apache.org/jira/browse/KAFKA-384
Project: Kafka
Issue Type: Bug
Affects Versions: 0.8
Reporter: Joel Koshy
Seeing intermittent failures in 0.8 unit tests. Also, many sleeps can be removed (with producer acks in place) and I think MockTime isn't used in some places where it should.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (KAFKA-384) Fix intermittent test failures and
remove unnecessary sleeps
Posted by "Neha Narkhede (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Neha Narkhede reassigned KAFKA-384:
-----------------------------------
Assignee: Neha Narkhede
> Fix intermittent test failures and remove unnecessary sleeps
> ------------------------------------------------------------
>
> Key: KAFKA-384
> URL: https://issues.apache.org/jira/browse/KAFKA-384
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.8
> Reporter: Joel Koshy
> Assignee: Neha Narkhede
>
> Seeing intermittent failures in 0.8 unit tests. Also, many sleeps can be removed (with producer acks in place) and I think MockTime isn't used in some places where it should.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-384) Fix intermittent test failures and
remove unnecessary sleeps
Posted by "Jay Kreps (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13425274#comment-13425274 ]
Jay Kreps commented on KAFKA-384:
---------------------------------
You are my hero...
> Fix intermittent test failures and remove unnecessary sleeps
> ------------------------------------------------------------
>
> Key: KAFKA-384
> URL: https://issues.apache.org/jira/browse/KAFKA-384
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.8
> Reporter: Joel Koshy
> Assignee: Neha Narkhede
> Attachments: kafka-384-v1.patch, kafka-384-v2.patch
>
>
> Seeing intermittent failures in 0.8 unit tests. Also, many sleeps can be removed (with producer acks in place) and I think MockTime isn't used in some places where it should.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work started] (KAFKA-384) Fix intermittent test failures
and remove unnecessary sleeps
Posted by "Neha Narkhede (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on KAFKA-384 started by Neha Narkhede.
> Fix intermittent test failures and remove unnecessary sleeps
> ------------------------------------------------------------
>
> Key: KAFKA-384
> URL: https://issues.apache.org/jira/browse/KAFKA-384
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.8
> Reporter: Joel Koshy
> Assignee: Neha Narkhede
>
> Seeing intermittent failures in 0.8 unit tests. Also, many sleeps can be removed (with producer acks in place) and I think MockTime isn't used in some places where it should.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (KAFKA-384) Fix intermittent test failures
and remove unnecessary sleeps
Posted by "Neha Narkhede (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13424610#comment-13424610 ]
Neha Narkhede edited comment on KAFKA-384 at 7/29/12 7:31 PM:
--------------------------------------------------------------
This patch removes sleep statements from the unit tests. Changes include -
1. All sleep statements except those related to a scheduler are removed. So 2-3 sleep statements that exercise the producer side queue expiration logic have not been removed
2. For the Log tests, passed in a Time parameter to the Log. Kafka server already takes in an optional Time parameter that defaults to SystemTime. However, it wasn't passed around in LogManager. Fixed it so KafkaServer passes its Time
variable to LogManager which passes it to Log.
This is useful in removing all sleep statements from unit tests for the log manager and logs
was (Author: nehanarkhede):
This patch removes sleep statements from the unit tests. Changes include -
1. All sleep statements except those related to a scheduler are removed. This includes the producer side queue expiration logic
2. For the Log tests, passed in a Time parameter to the Log. Kafka server already takes in an optional Time parameter that defaults to SystemTime. However, it wasn't passed around in LogManager. Fixed it so KafkaServer passes its Time
variable to LogManager which passes it to Log.
This is useful in removing all sleep statements from unit tests for the log manager and logs
> Fix intermittent test failures and remove unnecessary sleeps
> ------------------------------------------------------------
>
> Key: KAFKA-384
> URL: https://issues.apache.org/jira/browse/KAFKA-384
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.8
> Reporter: Joel Koshy
> Assignee: Neha Narkhede
> Attachments: kafka-384-v1.patch
>
>
> Seeing intermittent failures in 0.8 unit tests. Also, many sleeps can be removed (with producer acks in place) and I think MockTime isn't used in some places where it should.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-384) Fix intermittent test failures and
remove unnecessary sleeps
Posted by "Neha Narkhede (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Neha Narkhede updated KAFKA-384:
--------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
Thanks all for the review! Committed the v2 patch.
> Fix intermittent test failures and remove unnecessary sleeps
> ------------------------------------------------------------
>
> Key: KAFKA-384
> URL: https://issues.apache.org/jira/browse/KAFKA-384
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.8
> Reporter: Joel Koshy
> Assignee: Neha Narkhede
> Attachments: kafka-384-v1.patch, kafka-384-v2.patch
>
>
> Seeing intermittent failures in 0.8 unit tests. Also, many sleeps can be removed (with producer acks in place) and I think MockTime isn't used in some places where it should.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-384) Fix intermittent test failures and
remove unnecessary sleeps
Posted by "Neha Narkhede (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13425985#comment-13425985 ]
Neha Narkhede commented on KAFKA-384:
-------------------------------------
KAFKA-343 checkin broke some unit tests and cause others to hang. I think I might have to hold off on the checkin until that is either fixed or reverted.
> Fix intermittent test failures and remove unnecessary sleeps
> ------------------------------------------------------------
>
> Key: KAFKA-384
> URL: https://issues.apache.org/jira/browse/KAFKA-384
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.8
> Reporter: Joel Koshy
> Assignee: Neha Narkhede
> Attachments: kafka-384-v1.patch, kafka-384-v2.patch
>
>
> Seeing intermittent failures in 0.8 unit tests. Also, many sleeps can be removed (with producer acks in place) and I think MockTime isn't used in some places where it should.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-384) Fix intermittent test failures and
remove unnecessary sleeps
Posted by "Jun Rao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13425294#comment-13425294 ]
Jun Rao commented on KAFKA-384:
-------------------------------
+1 on v2 too.
> Fix intermittent test failures and remove unnecessary sleeps
> ------------------------------------------------------------
>
> Key: KAFKA-384
> URL: https://issues.apache.org/jira/browse/KAFKA-384
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.8
> Reporter: Joel Koshy
> Assignee: Neha Narkhede
> Attachments: kafka-384-v1.patch, kafka-384-v2.patch
>
>
> Seeing intermittent failures in 0.8 unit tests. Also, many sleeps can be removed (with producer acks in place) and I think MockTime isn't used in some places where it should.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-384) Fix intermittent test failures and
remove unnecessary sleeps
Posted by "Jun Rao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13425180#comment-13425180 ]
Jun Rao commented on KAFKA-384:
-------------------------------
Thanks for the patch. It looks good. Just 1 comment:
1. AsyncProducerTest.testQueueTimeExpired(): This is an existing issue, but probably can be fixed in this patch too. It seems that we should do producerSendThread.shutdown after EasyMock.verify(mockHandler). Shutdown always sends all remaining messages in the buffer. If we call shutdown before verify, it's not clear if the send was triggered by timeout or shutdown.
> Fix intermittent test failures and remove unnecessary sleeps
> ------------------------------------------------------------
>
> Key: KAFKA-384
> URL: https://issues.apache.org/jira/browse/KAFKA-384
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.8
> Reporter: Joel Koshy
> Assignee: Neha Narkhede
> Attachments: kafka-384-v1.patch
>
>
> Seeing intermittent failures in 0.8 unit tests. Also, many sleeps can be removed (with producer acks in place) and I think MockTime isn't used in some places where it should.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-384) Fix intermittent test failures and
remove unnecessary sleeps
Posted by "Neha Narkhede (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Neha Narkhede updated KAFKA-384:
--------------------------------
Status: Patch Available (was: In Progress)
> Fix intermittent test failures and remove unnecessary sleeps
> ------------------------------------------------------------
>
> Key: KAFKA-384
> URL: https://issues.apache.org/jira/browse/KAFKA-384
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.8
> Reporter: Joel Koshy
> Assignee: Neha Narkhede
> Attachments: kafka-384-v1.patch
>
>
> Seeing intermittent failures in 0.8 unit tests. Also, many sleeps can be removed (with producer acks in place) and I think MockTime isn't used in some places where it should.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-384) Fix intermittent test failures and
remove unnecessary sleeps
Posted by "Joel Koshy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13425221#comment-13425221 ]
Joel Koshy commented on KAFKA-384:
----------------------------------
+1 for v2.
> Fix intermittent test failures and remove unnecessary sleeps
> ------------------------------------------------------------
>
> Key: KAFKA-384
> URL: https://issues.apache.org/jira/browse/KAFKA-384
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.8
> Reporter: Joel Koshy
> Assignee: Neha Narkhede
> Attachments: kafka-384-v1.patch, kafka-384-v2.patch
>
>
> Seeing intermittent failures in 0.8 unit tests. Also, many sleeps can be removed (with producer acks in place) and I think MockTime isn't used in some places where it should.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-384) Fix intermittent test failures and
remove unnecessary sleeps
Posted by "Neha Narkhede (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Neha Narkhede updated KAFKA-384:
--------------------------------
Attachment: kafka-384-v2.patch
Thanks for the review, Jun. That is a good point. I fixed that and also removed the reference to mockTime since that is not useful here.
> Fix intermittent test failures and remove unnecessary sleeps
> ------------------------------------------------------------
>
> Key: KAFKA-384
> URL: https://issues.apache.org/jira/browse/KAFKA-384
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.8
> Reporter: Joel Koshy
> Assignee: Neha Narkhede
> Attachments: kafka-384-v1.patch, kafka-384-v2.patch
>
>
> Seeing intermittent failures in 0.8 unit tests. Also, many sleeps can be removed (with producer acks in place) and I think MockTime isn't used in some places where it should.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-384) Fix intermittent test failures and
remove unnecessary sleeps
Posted by "Neha Narkhede (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Neha Narkhede updated KAFKA-384:
--------------------------------
Attachment: kafka-384-v1.patch
This patch removes sleep statements from the unit tests. Changes include -
1. All sleep statements except those related to a scheduler are removed. This includes the producer side queue expiration logic
2. For the Log tests, passed in a Time parameter to the Log. Kafka server already takes in an optional Time parameter that defaults to SystemTime. However, it wasn't passed around in LogManager. Fixed it so KafkaServer passes its Time
variable to LogManager which passes it to Log.
This is useful in removing all sleep statements from unit tests for the log manager and logs
> Fix intermittent test failures and remove unnecessary sleeps
> ------------------------------------------------------------
>
> Key: KAFKA-384
> URL: https://issues.apache.org/jira/browse/KAFKA-384
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.8
> Reporter: Joel Koshy
> Assignee: Neha Narkhede
> Attachments: kafka-384-v1.patch
>
>
> Seeing intermittent failures in 0.8 unit tests. Also, many sleeps can be removed (with producer acks in place) and I think MockTime isn't used in some places where it should.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira