[jira] [Commented] (KAFKA-6332) Kafka system tests should use nc instead of log grep to detect start-up

["Attila Sasvari (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-6332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16368328#comment-16368328 ] 

Attila Sasvari commented on KAFKA-6332:
---------------------------------------

[~ijuma] I'd like to work on this task, can you please assign it to me?

In {{kafka.py}}, we could add something similar to the [listening() function in zookeeper.py|https://github.com/apache/kafka/blob/ee352be9c88663b95b1096b7a294e61857857380/tests/kafkatest/services/zookeeper.py#L87] :
{code}
def listening(self, node, port):
      try:
          cmd = "nc -z %s %s" % (node.account.hostname, port)
          node.account.ssh_output(cmd, allow_fail=False)
          self.logger.debug("Kafka server started accepting connections at: '%s:%s')", node.account.hostname, port)
          return True
      except (RemoteCommandError, ValueError) as e:
          return False
{code}
and call it in {{start_node}}  
{code}
    def start_node(self, node):
...
        self.logger.debug("Attempting to start KafkaService on %s with command: %s" % (str(node.account), cmd))
        node.account.ssh(cmd)
        wait_until(lambda: self.listening(node, self.jmx_port), timeout_sec=30, backoff_sec=.25,
                   err_msg="Kafka server didn't finish startup")
{code}
I have a [work in progress|https://github.com/asasvari/kafka/commit/916ab0b8291c7db2d16d84fc847630972628e345], executed a few tests ({{./tests/kafkatest/tests/core/consumer_group_command_test.py}}, {{./tests/kafkatest/tests/core/simple_consumer_shell_test.py}}) and they passed.

However, {{listening()}} could be more general; so it might go into the {{utils}} module. I see there are references to {{monitor.wait_until}} grepping for messages in server logs at other places (like {{minikdc.py}}, {{connect.py}}, {{torgdor.py}}, {{streams.py}} - we need to know the exact ports).

> Kafka system tests should use nc instead of log grep to detect start-up
> -----------------------------------------------------------------------
>
>                 Key: KAFKA-6332
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6332
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Ismael Juma
>            Priority: Major
>              Labels: newbie
>
> [~ewencp] suggested using nc -z test instead of grepping the logs for a more reliable test. This came up when the system tests were broken by a log improvement change.
> Reference: https://github.com/apache/kafka/pull/3834



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)