You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Andriy Sorokhtey <so...@gmail.com> on 2018/05/08 12:01:28 UTC
Kafka system tests contribution
Hello Kafka team
I’d like to contribute to the Kafka system tests.
I’ve tried to execute system tests locally and I have some issues. Can
anyone give me a hand to figure out what’s wrong?
So, I see that multiple system tests are failing when I try to run it with
the docker or with vagrant.
Maybe there is some way to debug it using PyCharm. For example, put some
breakpoint and start debugging, when the test goes to the breakpoint I’d
like to go to instances and check what’s going on there.
I’ll be thankful for any advice.
Here is an example of one test failure:
> [INFO:2018-05-03 06:37:19,861]: Triggering test 1 of 37...
> [INFO:2018-05-03 06:37:19,870]: RunnerClient: Loading test {'directory':
> '/opt/kafka-dev/tests/kafkatest/tests/streams', 'file_name':
> 'streams_broker_compatibility_test.py', 'method_name':
> 'test_compatible_brokers_eos_disabled', 'cls_name':
> 'StreamsBrokerCompatibility', 'injected_args': {'broker_version':
> '0.10.1.1'}}
> [INFO:2018-05-03 06:37:19,874]: RunnerClient:
> kafkatest.tests.streams.streams_broker_compatibility_test.StreamsBrokerCompatibility.test_compatible_brokers_eos_disabled.broker_version=
> 0.10.1.1: Setting up...
> [INFO:2018-05-03 06:37:22,484]: RunnerClient:
> kafkatest.tests.streams.streams_broker_compatibility_test.StreamsBrokerCompatibility.test_compatible_brokers_eos_disabled.broker_version=
> 0.10.1.1: Running...
> [INFO:2018-05-03 06:38:34,129]: RunnerClient:
> kafkatest.tests.streams.streams_broker_compatibility_test.StreamsBrokerCompatibility.test_compatible_brokers_eos_disabled.broker_version=
> 0.10.1.1: FAIL: Never saw message indicating StreamsTest finished startup
> on ducker@ducker05
> Traceback (most recent call last):
> File
> "/usr/local/lib/python2.7/dist-packages/ducktape/tests/runner_client.py",
> line 132, in run
> data = self.run_test()
> File
> "/usr/local/lib/python2.7/dist-packages/ducktape/tests/runner_client.py",
> line 185, in run_test
> return self.test_context.function(self.test)
> File "/usr/local/lib/python2.7/dist-packages/ducktape/mark/_mark.py", line
> 324, in wrapper
> return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
> File
> "/opt/kafka-dev/tests/kafkatest/tests/streams/streams_broker_compatibility_test.py",
> line 84, in test_compatible_brokers_eos_disabled
> processor.start()
> File
> "/usr/local/lib/python2.7/dist-packages/ducktape/services/service.py", line
> 234, in start
> self.start_node(node)
> File "/opt/kafka-dev/tests/kafkatest/services/streams.py", line 138, in
> start_node
> monitor.wait_until('StreamsTest instance started', timeout_sec=60,
> err_msg="Never saw message indicating StreamsTest finished startup on " +
> str(node.account))
> File
> "/usr/local/lib/python2.7/dist-packages/ducktape/cluster/remoteaccount.py",
> line 668, in wait_until
> allow_fail=True) == 0, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/ducktape/utils/util.py", line
> 36, in wait_until
> raise TimeoutError(err_msg)
> TimeoutError: Never saw message indicating StreamsTest finished startup on
> ducker@ducker05
If I figure out what's wrong I can try to fix other tests.
--
*Sincerely*
*Andriy Sorokhtey*
+380681890146
Re: Kafka system tests contribution
Posted by "Matthias J. Sax" <ma...@confluent.io>.
Hard to say why it fails. Note, that some system tests a flaky (we are
working constantly to stabilize them)
What helps debugging is to look into the ducktape log itself. You can
find it in the file `test_log.debug` that is written into
`resutls/<data>/<testClassName>/<testMethodName>/<counter>/`
Also inspect the actual test log files, as well as stdout and stderr files.
Unfortunately, I am not aware of a technique that allows you to set a
break point in a system test.
-Matthias
On 5/8/18 5:01 AM, Andriy Sorokhtey wrote:
> Hello Kafka team
>
> I’d like to contribute to the Kafka system tests.
>
> I’ve tried to execute system tests locally and I have some issues. Can
> anyone give me a hand to figure out what’s wrong?
>
> So, I see that multiple system tests are failing when I try to run it with
> the docker or with vagrant.
> Maybe there is some way to debug it using PyCharm. For example, put some
> breakpoint and start debugging, when the test goes to the breakpoint I’d
> like to go to instances and check what’s going on there.
> I’ll be thankful for any advice.
>
> Here is an example of one test failure:
>
>> [INFO:2018-05-03 06:37:19,861]: Triggering test 1 of 37...
>> [INFO:2018-05-03 06:37:19,870]: RunnerClient: Loading test {'directory':
>> '/opt/kafka-dev/tests/kafkatest/tests/streams', 'file_name':
>> 'streams_broker_compatibility_test.py', 'method_name':
>> 'test_compatible_brokers_eos_disabled', 'cls_name':
>> 'StreamsBrokerCompatibility', 'injected_args': {'broker_version':
>> '0.10.1.1'}}
>> [INFO:2018-05-03 06:37:19,874]: RunnerClient:
>> kafkatest.tests.streams.streams_broker_compatibility_test.StreamsBrokerCompatibility.test_compatible_brokers_eos_disabled.broker_version=
>> 0.10.1.1: Setting up...
>> [INFO:2018-05-03 06:37:22,484]: RunnerClient:
>> kafkatest.tests.streams.streams_broker_compatibility_test.StreamsBrokerCompatibility.test_compatible_brokers_eos_disabled.broker_version=
>> 0.10.1.1: Running...
>> [INFO:2018-05-03 06:38:34,129]: RunnerClient:
>> kafkatest.tests.streams.streams_broker_compatibility_test.StreamsBrokerCompatibility.test_compatible_brokers_eos_disabled.broker_version=
>> 0.10.1.1: FAIL: Never saw message indicating StreamsTest finished startup
>> on ducker@ducker05
>> Traceback (most recent call last):
>> File
>> "/usr/local/lib/python2.7/dist-packages/ducktape/tests/runner_client.py",
>> line 132, in run
>> data = self.run_test()
>> File
>> "/usr/local/lib/python2.7/dist-packages/ducktape/tests/runner_client.py",
>> line 185, in run_test
>> return self.test_context.function(self.test)
>> File "/usr/local/lib/python2.7/dist-packages/ducktape/mark/_mark.py", line
>> 324, in wrapper
>> return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
>> File
>> "/opt/kafka-dev/tests/kafkatest/tests/streams/streams_broker_compatibility_test.py",
>> line 84, in test_compatible_brokers_eos_disabled
>> processor.start()
>> File
>> "/usr/local/lib/python2.7/dist-packages/ducktape/services/service.py", line
>> 234, in start
>> self.start_node(node)
>> File "/opt/kafka-dev/tests/kafkatest/services/streams.py", line 138, in
>> start_node
>> monitor.wait_until('StreamsTest instance started', timeout_sec=60,
>> err_msg="Never saw message indicating StreamsTest finished startup on " +
>> str(node.account))
>> File
>> "/usr/local/lib/python2.7/dist-packages/ducktape/cluster/remoteaccount.py",
>> line 668, in wait_until
>> allow_fail=True) == 0, **kwargs)
>> File "/usr/local/lib/python2.7/dist-packages/ducktape/utils/util.py", line
>> 36, in wait_until
>> raise TimeoutError(err_msg)
>> TimeoutError: Never saw message indicating StreamsTest finished startup on
>> ducker@ducker05
>
>
> If I figure out what's wrong I can try to fix other tests.
>
Re: Kafka system tests contribution
Posted by Colin McCabe <cm...@apache.org>.
Hi Andriy,
Try looking at the logs to see why the test failed.
best,
Colin
On Wed, May 16, 2018, at 07:08, Andriy Sorokhtey wrote:
> Hi,
>
> Did anyone had a chance to take a look at this issue?
>
> 2018-05-08 15:01 GMT+03:00 Andriy Sorokhtey <so...@gmail.com>:
>
> > Hello Kafka team
> >
> > I’d like to contribute to the Kafka system tests.
> >
> > I’ve tried to execute system tests locally and I have some issues. Can
> > anyone give me a hand to figure out what’s wrong?
> >
> > So, I see that multiple system tests are failing when I try to run it with
> > the docker or with vagrant.
> > Maybe there is some way to debug it using PyCharm. For example, put some
> > breakpoint and start debugging, when the test goes to the breakpoint I’d
> > like to go to instances and check what’s going on there.
> > I’ll be thankful for any advice.
> >
> > Here is an example of one test failure:
> >
> >> [INFO:2018-05-03 06:37:19,861]: Triggering test 1 of 37...
> >> [INFO:2018-05-03 06:37:19,870]: RunnerClient: Loading test {'directory':
> >> '/opt/kafka-dev/tests/kafkatest/tests/streams', 'file_name':
> >> 'streams_broker_compatibility_test.py', 'method_name':
> >> 'test_compatible_brokers_eos_disabled', 'cls_name':
> >> 'StreamsBrokerCompatibility', 'injected_args': {'broker_version':
> >> '0.10.1.1'}}
> >> [INFO:2018-05-03 06:37:19,874]: RunnerClient: kafkatest.tests.streams.
> >> streams_broker_compatibility_test.StreamsBrokerCompatibility.
> >> test_compatible_brokers_eos_disabled.broker_version=0.10.1.1: Setting
> >> up...
> >> [INFO:2018-05-03 06:37:22,484]: RunnerClient: kafkatest.tests.streams.
> >> streams_broker_compatibility_test.StreamsBrokerCompatibility.
> >> test_compatible_brokers_eos_disabled.broker_version=0.10.1.1: Running...
> >> [INFO:2018-05-03 06:38:34,129]: RunnerClient: kafkatest.tests.streams.
> >> streams_broker_compatibility_test.StreamsBrokerCompatibility.
> >> test_compatible_brokers_eos_disabled.broker_version=0.10.1.1: FAIL:
> >> Never saw message indicating StreamsTest finished startup on ducker@ducker05
> >> Traceback (most recent call last):
> >> File "/usr/local/lib/python2.7/dist-packages/ducktape/tests/runner_client.py",
> >> line 132, in run
> >> data = self.run_test()
> >> File "/usr/local/lib/python2.7/dist-packages/ducktape/tests/runner_client.py",
> >> line 185, in run_test
> >> return self.test_context.function(self.test)
> >> File "/usr/local/lib/python2.7/dist-packages/ducktape/mark/_mark.py",
> >> line 324, in wrapper
> >> return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
> >> File "/opt/kafka-dev/tests/kafkatest/tests/streams/
> >> streams_broker_compatibility_test.py", line 84, in
> >> test_compatible_brokers_eos_disabled
> >> processor.start()
> >> File "/usr/local/lib/python2.7/dist-packages/ducktape/services/service.py",
> >> line 234, in start
> >> self.start_node(node)
> >> File "/opt/kafka-dev/tests/kafkatest/services/streams.py", line 138, in
> >> start_node
> >> monitor.wait_until('StreamsTest instance started', timeout_sec=60,
> >> err_msg="Never saw message indicating StreamsTest finished startup on " +
> >> str(node.account))
> >> File "/usr/local/lib/python2.7/dist-packages/ducktape/cluster/remoteaccount.py",
> >> line 668, in wait_until
> >> allow_fail=True) == 0, **kwargs)
> >> File "/usr/local/lib/python2.7/dist-packages/ducktape/utils/util.py",
> >> line 36, in wait_until
> >> raise TimeoutError(err_msg)
> >> TimeoutError: Never saw message indicating StreamsTest finished startup
> >> on ducker@ducker05
> >
> >
> > If I figure out what's wrong I can try to fix other tests.
> >
> > --
> >
> > *Sincerely*
> > *Andriy Sorokhtey*
> > +380681890146
> >
>
>
>
> --
>
> *Sincerely*
> *Andriy Sorokhtey*
> +380681890146
Re: Kafka system tests contribution
Posted by Andriy Sorokhtey <so...@gmail.com>.
Hi,
Did anyone had a chance to take a look at this issue?
2018-05-08 15:01 GMT+03:00 Andriy Sorokhtey <so...@gmail.com>:
> Hello Kafka team
>
> I’d like to contribute to the Kafka system tests.
>
> I’ve tried to execute system tests locally and I have some issues. Can
> anyone give me a hand to figure out what’s wrong?
>
> So, I see that multiple system tests are failing when I try to run it with
> the docker or with vagrant.
> Maybe there is some way to debug it using PyCharm. For example, put some
> breakpoint and start debugging, when the test goes to the breakpoint I’d
> like to go to instances and check what’s going on there.
> I’ll be thankful for any advice.
>
> Here is an example of one test failure:
>
>> [INFO:2018-05-03 06:37:19,861]: Triggering test 1 of 37...
>> [INFO:2018-05-03 06:37:19,870]: RunnerClient: Loading test {'directory':
>> '/opt/kafka-dev/tests/kafkatest/tests/streams', 'file_name':
>> 'streams_broker_compatibility_test.py', 'method_name':
>> 'test_compatible_brokers_eos_disabled', 'cls_name':
>> 'StreamsBrokerCompatibility', 'injected_args': {'broker_version':
>> '0.10.1.1'}}
>> [INFO:2018-05-03 06:37:19,874]: RunnerClient: kafkatest.tests.streams.
>> streams_broker_compatibility_test.StreamsBrokerCompatibility.
>> test_compatible_brokers_eos_disabled.broker_version=0.10.1.1: Setting
>> up...
>> [INFO:2018-05-03 06:37:22,484]: RunnerClient: kafkatest.tests.streams.
>> streams_broker_compatibility_test.StreamsBrokerCompatibility.
>> test_compatible_brokers_eos_disabled.broker_version=0.10.1.1: Running...
>> [INFO:2018-05-03 06:38:34,129]: RunnerClient: kafkatest.tests.streams.
>> streams_broker_compatibility_test.StreamsBrokerCompatibility.
>> test_compatible_brokers_eos_disabled.broker_version=0.10.1.1: FAIL:
>> Never saw message indicating StreamsTest finished startup on ducker@ducker05
>> Traceback (most recent call last):
>> File "/usr/local/lib/python2.7/dist-packages/ducktape/tests/runner_client.py",
>> line 132, in run
>> data = self.run_test()
>> File "/usr/local/lib/python2.7/dist-packages/ducktape/tests/runner_client.py",
>> line 185, in run_test
>> return self.test_context.function(self.test)
>> File "/usr/local/lib/python2.7/dist-packages/ducktape/mark/_mark.py",
>> line 324, in wrapper
>> return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
>> File "/opt/kafka-dev/tests/kafkatest/tests/streams/
>> streams_broker_compatibility_test.py", line 84, in
>> test_compatible_brokers_eos_disabled
>> processor.start()
>> File "/usr/local/lib/python2.7/dist-packages/ducktape/services/service.py",
>> line 234, in start
>> self.start_node(node)
>> File "/opt/kafka-dev/tests/kafkatest/services/streams.py", line 138, in
>> start_node
>> monitor.wait_until('StreamsTest instance started', timeout_sec=60,
>> err_msg="Never saw message indicating StreamsTest finished startup on " +
>> str(node.account))
>> File "/usr/local/lib/python2.7/dist-packages/ducktape/cluster/remoteaccount.py",
>> line 668, in wait_until
>> allow_fail=True) == 0, **kwargs)
>> File "/usr/local/lib/python2.7/dist-packages/ducktape/utils/util.py",
>> line 36, in wait_until
>> raise TimeoutError(err_msg)
>> TimeoutError: Never saw message indicating StreamsTest finished startup
>> on ducker@ducker05
>
>
> If I figure out what's wrong I can try to fix other tests.
>
> --
>
> *Sincerely*
> *Andriy Sorokhtey*
> +380681890146
>
--
*Sincerely*
*Andriy Sorokhtey*
+380681890146