You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "ahuang98 (via GitHub)" <gi...@apache.org> on 2024/09/26 21:57:29 UTC
[PR] MINOR: Fix kafkatest advertised listeners [kafka]
ahuang98 opened a new pull request, #17294:
URL: https://github.com/apache/kafka/pull/17294
Followup for https://github.com/apache/kafka/pull/17146
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Fix kafkatest advertised listeners [kafka]
Posted by "ahuang98 (via GitHub)" <gi...@apache.org>.
ahuang98 commented on PR #17294:
URL: https://github.com/apache/kafka/pull/17294#issuecomment-2379874423
@bbejeck what are the tests you are trying to run/does this change resolve your test failures?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Fix kafkatest advertised listeners [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #17294:
URL: https://github.com/apache/kafka/pull/17294#issuecomment-2378033264
I encountered the following error locally with this patch. @ahuang98 Could you please take a look?
```
rror("'KafkaService' object has no attribute 'dynamicRaftQuorum'")
Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", line 186, in _do_run
data = self.run_test()
File "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", line 246, in run_test
return self.test_context.function(self.test)
File "/usr/local/lib/python3.9/dist-packages/ducktape/mark/_mark.py", line 433, in wrapper
return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
File "/opt/kafka-dev/tests/kafkatest/tests/client/message_format_change_test.py", line 88, in test_compatibility
self.kafka.start()
File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 674, in start
Service.start(self, **kwargs)
File "/usr/local/lib/python3.9/dist-packages/ducktape/services/service.py", line 265, in start
self.start_node(node, **kwargs)
File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 892, in start_node
prop_file = self.prop_file(node)
File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 772, in prop_file
self.set_protocol_and_port(node)
File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 755, in set_protocol_and_port
if (self.dynamicRaftQuorum and quorum.NodeQuorumInfo(self.quorum_info, node).has_controller_role) or \
AttributeError: 'KafkaService' object has no attribute 'dynamicRaftQuorum'
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Fix kafkatest advertised listeners [kafka]
Posted by "ahuang98 (via GitHub)" <gi...@apache.org>.
ahuang98 commented on PR #17294:
URL: https://github.com/apache/kafka/pull/17294#issuecomment-2379884106
@chia7712 tried running message_format_change_test.py locally and it is passing with this change
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Fix kafkatest advertised listeners [kafka]
Posted by "ahuang98 (via GitHub)" <gi...@apache.org>.
ahuang98 commented on PR #17294:
URL: https://github.com/apache/kafka/pull/17294#issuecomment-2380249154
figured out the issue, another error in the new list comprehension 😅 in favor of code clarity, I'm going to opt for some code duplication
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Fix kafkatest advertised listeners [kafka]
Posted by "bbejeck (via GitHub)" <gi...@apache.org>.
bbejeck commented on PR #17294:
URL: https://github.com/apache/kafka/pull/17294#issuecomment-2379918654
@ahuang98 I still get the following error
``` RemoteCommandError({'ssh_config': {'host': 'ducker03', 'hostname': 'ducker03', 'user': 'ducker', 'port': 22, 'password': '', 'identityfile': '/home/ducker/.ssh/id_rsa', 'connecttimeout': None}, 'hostname': 'ducker03', 'ssh_hostname': 'ducker03', 'user': 'ducker', 'externally_routable_ip': 'ducker03', '_logger': <Logger kafkatest.tests.streams.streams_optimized_test.StreamsOptimizedTest.test_upgrade_optimized_topology.metadata_quorum=COMBINED_KRAFT-1 (DEBUG)>, 'os': 'linux', '_ssh_client': <paramiko.client.SSHClient object at 0xffff8e783e50>, '_sftp_client': <paramiko.sftp_client.SFTPClient object at 0xffff8e86ec10>, '_custom_ssh_exception_checks': None}, '/opt/kafka-dev/bin/kafka-storage.sh format --ignore-formatted --config /mnt/kafka/kafka.properties --cluster-id I2eXt9rvSnyhct8BYmW6-w', 1, b'Exception in thread "main" java.lang.IllegalArgumentException: requirement failed: If process.roles contains just the \'broker\' role, the node id 2 must not be included in the set of v
oters controller.quorum.voters=Set(2)\n\tat scala.Predef$.require(Predef.scala:337)\n\tat kafka.server.KafkaConfig.validateValues(KafkaConfig.scala:921)\n\tat kafka.server.KafkaConfig.<init>(KafkaConfig.scala:830)\n\tat kafka.server.KafkaConfig.<init>(KafkaConfig.scala:185)\n\tat kafka.tools.StorageTool$.$anonfun$execute$1(StorageTool.scala:80)\n\tat scala.Option.flatMap(Option.scala:283)\n\tat kafka.tools.StorageTool$.execute(StorageTool.scala:80)\n\tat kafka.tools.StorageTool$.main(StorageTool.scala:47)\n\tat kafka.tools.StorageTool.main(StorageTool.scala)\n')
Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", line 186, in _do_run
data = self.run_test()
File "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", line 246, in run_test
return self.test_context.function(self.test)
File "/usr/local/lib/python3.9/dist-packages/ducktape/mark/_mark.py", line 433, in wrapper
return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
File "/opt/kafka-dev/tests/kafkatest/tests/streams/streams_optimized_test.py", line 61, in test_upgrade_optimized_topology
self.kafka.start()
File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 675, in start
Service.start(self, **kwargs)
File "/usr/local/lib/python3.9/dist-packages/ducktape/services/service.py", line 265, in start
self.start_node(node, **kwargs)
File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 909, in start_node
node.account.ssh(cmd)
File "/usr/local/lib/python3.9/dist-packages/ducktape/cluster/remoteaccount.py", line 35, in wrapper
return method(self, *args, **kwargs)
File "/usr/local/lib/python3.9/dist-packages/ducktape/cluster/remoteaccount.py", line 310, in ssh
raise RemoteCommandError(self, cmd, exit_status, stderr.read())
ducktape.cluster.remoteaccount.RemoteCommandError: ducker@ducker03: Command '/opt/kafka-dev/bin/kafka-storage.sh format --ignore-formatted --config /mnt/kafka/kafka.properties --cluster-id I2eXt9rvSnyhct8BYmW6-w' returned non-zero exit status 1. Remote error message: b'Exception in thread "main" java.lang.IllegalArgumentException: requirement failed: If process.roles contains just the \'broker\' role, the node id 2 must not be included in the set of voters controller.quorum.voters=Set(2)\n\tat scala.Predef$.require(Predef.scala:337)\n\tat kafka.server.KafkaConfig.validateValues(KafkaConfig.scala:921)\n\tat kafka.server.KafkaConfig.<init>(KafkaConfig.scala:830)\n\tat kafka.server.KafkaConfig.<init>(KafkaConfig.scala:185)\n\tat kafka.tools.StorageTool$.$anonfun$execute$1(StorageTool.scala:80)\n\tat scala.Option.flatMap(Option.scala:283)\n\tat kafka.tools.StorageTool$.execute(StorageTool.scala:80)\n\tat kafka.tools.StorageTool$.main(StorageTool.scala:47)\n\tat kafka.tools.StorageTool
.main(StorageTool.scala)\n'
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Fix kafkatest advertised listeners [kafka]
Posted by "ahuang98 (via GitHub)" <gi...@apache.org>.
ahuang98 commented on PR #17294:
URL: https://github.com/apache/kafka/pull/17294#issuecomment-2379988457
@bbejeck I'm not seeing that error locally, I get the following with no obvious errors in the logs
```
TimeoutError("Kafka server didn't finish startup in 60 seconds")
Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", line 186, in _do_run
data = self.run_test()
File "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", line 246, in run_test
return self.test_context.function(self.test)
File "/usr/local/lib/python3.9/dist-packages/ducktape/mark/_mark.py", line 433, in wrapper
return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
File "/opt/kafka-dev/tests/kafkatest/tests/streams/streams_optimized_test.py", line 69, in test_upgrade_optimized_topology
self.kafka.start()
File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 682, in start
self.wait_for_start(node, monitor, timeout_sec)
File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line 925, in wait_for_start
monitor.wait_until("Kafka\s*Server.*started", timeout_sec=timeout_sec, backoff_sec=.25,
File "/usr/local/lib/python3.9/dist-packages/ducktape/cluster/remoteaccount.py", line 753, in wait_until
return wait_until(lambda: self.acct.ssh("tail -c +%d %s | grep '%s'" % (self.offset + 1, self.log, pattern),
File "/usr/local/lib/python3.9/dist-packages/ducktape/utils/util.py", line 58, in wait_until
raise TimeoutError(err_msg() if callable(err_msg) else err_msg) from last_exception
ducktape.errors.TimeoutError: Kafka server didn't finish startup in 60 seconds
```
just confirming - do you see that error when checking out just this PR branch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Fix kafkatest advertised listeners [kafka]
Posted by "bbejeck (via GitHub)" <gi...@apache.org>.
bbejeck commented on PR #17294:
URL: https://github.com/apache/kafka/pull/17294#issuecomment-2380082805
@ahuang98
For this KafkaService config
```
self.kafka = KafkaService(self.test_context, num_nodes=3, controller_num_nodes_override=1,
zk=None, topics=self.topics, dynamicRaftQuorum=False)
```
I get the error I've posted before
```
File "/usr/local/lib/python3.9/dist-packages/ducktape/cluster/remoteaccount.py", line 310, in ssh
raise RemoteCommandError(self, cmd, exit_status, stderr.read())
ducktape.cluster.remoteaccount.RemoteCommandError:
ducker@ducker03: Command '/opt/kafka-dev/bin/kafka-storage.sh format --ignore-formatted --config /mnt/kafka/kafka.properties --cluster-id I2eXt9rvSnyhct8BYmW6-w' returned non-zero exit status 1. Remote error message: b'Exception in thread "main" java.lang.IllegalArgumentException: requirement failed:
If process.roles contains just the 'broker' role, the node id 2 must not be included in the set of voters controller.quorum.voters=Set(2)\n\tat
```
If I update it to have a combined controller+broker per node
```
self.kafka = KafkaService(self.test_context, num_nodes=3,
zk=None, topics=self.topics, dynamicRaftQuorum=False)
```
I get this
```
ducktape.cluster.remoteaccount.RemoteCommandError: ducker@ducker02: Command '/opt/kafka-dev/bin/kafka-topics.sh --bootstrap-server ducker02:9092,ducker03:9092,ducker04:9092 --create --topic inputTopic --partitions 6 --replication-factor 1' returned non-zero exit status 1.
Remote error message: b'[2024-09-27 21:05:38,441]
ERROR org.apache.kafka.common.errors.TimeoutException:
Timed out waiting for a node assignment. Call: createTopics\n (org.apache.kafka.tools.TopicCommand)\n'
```
I also tried using the `isolated_kafka` mode and when I did that I received the `Kafka server didn't finish startup in 60 second` error
HTH
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Fix kafkatest advertised listeners [kafka]
Posted by "bbejeck (via GitHub)" <gi...@apache.org>.
bbejeck commented on PR #17294:
URL: https://github.com/apache/kafka/pull/17294#issuecomment-2379891870
@ahuang98 the tests under `tests/kafkatest/tests/streams` I'll update trunk and re-run and let you know.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Fix kafkatest advertised listeners [kafka]
Posted by "ahuang98 (via GitHub)" <gi...@apache.org>.
ahuang98 commented on PR #17294:
URL: https://github.com/apache/kafka/pull/17294#issuecomment-2378039416
@chia7712 thanks for bringing that up, non-kraft mode tests would likely fail since dynamicRaftQuorum wasn't initialized in that case
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Fix kafkatest advertised listeners [kafka]
Posted by "ahuang98 (via GitHub)" <gi...@apache.org>.
ahuang98 commented on PR #17294:
URL: https://github.com/apache/kafka/pull/17294#issuecomment-2378086787
having issues with the branch builder :( it worked for me yesterday but not today
```
paramiko.ssh_exception.SSHException: encountered RSA key, expected OPENSSH key
```
https://semaphore.ci.confluent.io/jobs/104b6698-910a-4958-9247-a759cb786c0d
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Fix kafkatest advertised listeners [kafka]
Posted by "bbejeck (via GitHub)" <gi...@apache.org>.
bbejeck commented on PR #17294:
URL: https://github.com/apache/kafka/pull/17294#issuecomment-2380715345
@ahuang98 - that did it! Thanks for your efforts to get this fixed, this test was failing previously without this fix
```
[INFO:2024-09-28 08:46:22,475]: RunnerClient: Loading test {'directory': '/opt/kafka-dev/tests/kafkatest/tests/streams', 'file_name': 'streams_optimized_test.py', 'cls_name': 'StreamsOptimizedTest', 'method_name': 'test_upgrade_optimized_topology', 'injected_args': {'metadata_quorum': 'COMBINED_KRAFT'}}
[INFO:2024-09-28 08:46:22,478]: RunnerClient: kafkatest.tests.streams.streams_optimized_test.StreamsOptimizedTest.test_upgrade_optimized_topology.metadata_quorum=COMBINED_KRAFT: on run 1/1
[INFO:2024-09-28 08:46:22,480]: RunnerClient: kafkatest.tests.streams.streams_optimized_test.StreamsOptimizedTest.test_upgrade_optimized_topology.metadata_quorum=COMBINED_KRAFT: Setting up...
[INFO:2024-09-28 08:46:22,480]: RunnerClient: kafkatest.tests.streams.streams_optimized_test.StreamsOptimizedTest.test_upgrade_optimized_topology.metadata_quorum=COMBINED_KRAFT: Running...
[INFO:2024-09-28 08:47:07,619]: RunnerClient: kafkatest.tests.streams.streams_optimized_test.StreamsOptimizedTest.test_upgrade_optimized_topology.metadata_quorum=COMBINED_KRAFT: Tearing down...
[INFO:2024-09-28 08:47:36,272]: RunnerClient: kafkatest.tests.streams.streams_optimized_test.StreamsOptimizedTest.test_upgrade_optimized_topology.metadata_quorum=COMBINED_KRAFT: PASS
[WARNING - 2024-09-28 08:47:36,273 - runner_client - log - lineno:294]: RunnerClient: kafkatest.tests.streams.streams_optimized_test.StreamsOptimizedTest.test_upgrade_optimized_topology.metadata_quorum=COMBINED_KRAFT: Test requested 9 nodes, used only 8
[WARNING:2024-09-28 08:47:36,273]: RunnerClient: kafkatest.tests.streams.streams_optimized_test.StreamsOptimizedTest.test_upgrade_optimized_topology.metadata_quorum=COMBINED_KRAFT: Test requested 9 nodes, used only 8
[INFO:2024-09-28 08:47:36,275]: RunnerClient: kafkatest.tests.streams.streams_optimized_test.StreamsOptimizedTest.test_upgrade_optimized_topology.metadata_quorum=COMBINED_KRAFT: Data: None
================================================================================
SESSION REPORT (ALL TESTS)
ducktape version: 0.11.4
session_id: 2024-09-28--001
run time: 1 minute 13.818 seconds
tests run: 1
passed: 1
flaky: 0
failed: 0
ignored: 0
================================================================================
test_id: kafkatest.tests.streams.streams_optimized_test.StreamsOptimizedTest.test_upgrade_optimized_topology.metadata_quorum=COMBINED_KRAFT
status: PASS
run time: 1 minute 13.795 seconds
--------------------------------------------------------------------------------
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Fix kafkatest advertised listeners [kafka]
Posted by "bbejeck (via GitHub)" <gi...@apache.org>.
bbejeck commented on PR #17294:
URL: https://github.com/apache/kafka/pull/17294#issuecomment-2380725487
Kicked off system test build - will post the results here later
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Fix kafkatest advertised listeners [kafka]
Posted by "bbejeck (via GitHub)" <gi...@apache.org>.
bbejeck commented on PR #17294:
URL: https://github.com/apache/kafka/pull/17294#issuecomment-2380835562
Getting this error running system tests on branch builder
```
paramiko.ssh_exception.SSHException: encountered RSA key, expected OPENSSH key27:12
make: *** [Makefile:172: branch-builder-run-system-tests] Error 1
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Fix kafkatest advertised listeners [kafka]
Posted by "ahuang98 (via GitHub)" <gi...@apache.org>.
ahuang98 commented on PR #17294:
URL: https://github.com/apache/kafka/pull/17294#issuecomment-2381046153
> Getting this error running system tests on branch builder
same :(
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Fix kafkatest advertised listeners [kafka]
Posted by "ahuang98 (via GitHub)" <gi...@apache.org>.
ahuang98 commented on PR #17294:
URL: https://github.com/apache/kafka/pull/17294#issuecomment-2379951796
@bbejeck thanks a bunch, taking a look!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org