You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "chia7712 (via GitHub)" <gi...@apache.org> on 2024/12/20 18:03:10 UTC
[PR] MINOR: print the error log if we can't list pids [kafka]
chia7712 opened a new pull request, #18286:
URL: https://github.com/apache/kafka/pull/18286
used to debug https://github.com/apache/kafka/pull/17373#issuecomment-2557355489
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: upgrade ubuntu from trusty to bionic [kafka]
Posted by "brandboat (via GitHub)" <gi...@apache.org>.
brandboat commented on PR #18286:
URL: https://github.com/apache/kafka/pull/18286#issuecomment-2560061584
> @jolshan the root cause is that the output of ps ax is truncated. see following output
I believe this issue is related to the 4096-byte limit of /proc/pid/cmdline, which was fixed in Linux version 4.2. Upgrading to Ubuntu 15.10 or later (which uses Linux v4.2) should resolve the issue.
https://stackoverflow.com/questions/57117038/how-to-get-the-maximum-value-for-ps-output-in-linux-ubuntu-redhat-centos
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: upgrade ubuntu from trusty to bionic [kafka]
Posted by "jolshan (via GitHub)" <gi...@apache.org>.
jolshan commented on PR #18286:
URL: https://github.com/apache/kafka/pull/18286#issuecomment-2560123253
@chia7712 that may work for the short term as well. I tried a first pass with the new instance type and it didn't work quite right yet. I will keep trying a few things in the meantime.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: upgrade ubuntu from trusty to bionic [kafka]
Posted by "brandboat (via GitHub)" <gi...@apache.org>.
brandboat commented on PR #18286:
URL: https://github.com/apache/kafka/pull/18286#issuecomment-2560503653
> @brandboat do you have free cycle to file a patch? I'd like to keep this PR as it is.
I'll file a PR today, thanks for bring this up!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: print the error log if we can't list pids [kafka]
Posted by "ijuma (via GitHub)" <gi...@apache.org>.
ijuma commented on PR #18286:
URL: https://github.com/apache/kafka/pull/18286#issuecomment-2557966632
Is this only to debug the issue or is it generally useful?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: print the error log if we can't list pids [kafka]
Posted by "jolshan (via GitHub)" <gi...@apache.org>.
jolshan commented on PR #18286:
URL: https://github.com/apache/kafka/pull/18286#issuecomment-2557841864
And unfortunately, looks like we didn't log this line. :(
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: upgrade ubuntu from trusty to bionic [kafka]
Posted by "jolshan (via GitHub)" <gi...@apache.org>.
jolshan commented on PR #18286:
URL: https://github.com/apache/kafka/pull/18286#issuecomment-2560087225
Thanks for the patch -- I had at one point tried a stab at upgrading ubuntu since 14 is already way past EOL. Makes sense that we should upgrade.
I think our infra unfortunately overrides the version set here, but I think I originally got stuck with the other changes in the base.sh files. Give me a moment to re-setup the testing on my end and I will let you know how it goes.
I do think this is generally the correct direction to go.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: upgrade ubuntu from trusty to bionic [kafka]
Posted by "jolshan (via GitHub)" <gi...@apache.org>.
jolshan commented on PR #18286:
URL: https://github.com/apache/kafka/pull/18286#issuecomment-2560083055
Taking a look. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: print the error log if we can't list pids [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #18286:
URL: https://github.com/apache/kafka/pull/18286#issuecomment-2558816147
the above comment is not correct :(
The true root cause is that the `grep -i kafka.Kafka` command can inadvertently match paths like `kafka/kafka`. Specifically, it matched `Dlog4j.configuration=file:/mnt/kafka/kafka-log4j.properties` even though the process name was truncated.
However, [#17373](https://github.com/apache/kafka/pull/17373) changes the log path from `/mnt/kafka/kafka-log4j.properties` to `/mnt/kafka/log4j.properties`. As a result, the grep command no longer matches any relevant lines, leading to the failure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: upgrade ubuntu from trusty to bionic [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #18286:
URL: https://github.com/apache/kafka/pull/18286#issuecomment-2560101333
> I think our infra unfortunately overrides the version set here, but I think I originally got stuck with the other changes in the base.sh files. Give me a moment to re-setup the testing on my end and I will let you know how it goes.
Another solution is to use jcmd instead of ps ax ([KAFKA-18343](https://issues.apache.org/jira/browse/KAFKA-18343)) to avoid encountering the truncation issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: upgrade ubuntu from trusty to bionic [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #18286:
URL: https://github.com/apache/kafka/pull/18286#issuecomment-2561562428
> I'm afraid that we have to leave it as it is.
yes, +1 to keep it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: upgrade ubuntu from trusty to bionic [kafka]
Posted by "jolshan (via GitHub)" <gi...@apache.org>.
jolshan commented on PR #18286:
URL: https://github.com/apache/kafka/pull/18286#issuecomment-2560324322
I tried a few difference instances but I'm seeing the kafka failed to start up in 60 seconds. However, this time it looks like a different issue related to the servers not finding the controller correctly. 🤔 I wonder if there is another issue here. I also don't see the pid error at all.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: upgrade ubuntu from trusty to bionic [kafka]
Posted by "jolshan (via GitHub)" <gi...@apache.org>.
jolshan commented on PR #18286:
URL: https://github.com/apache/kafka/pull/18286#issuecomment-2561306063
> @jolshan maybe there are broken commands after upgrading the OS version.
I think there could also be something on our infra side that I'm not setting up correctly. I requested some folks on our side to take a look and due to the holidays, it may be a while. I will look into the second solution (jcmd) now. :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: upgrade ubuntu from trusty to bionic [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #18286:
URL: https://github.com/apache/kafka/pull/18286#issuecomment-2560499043
@jolshan maybe there are broken commands after upgrading the OS version. We can try the solution of https://issues.apache.org/jira/browse/KAFKA-18343 which is more compatible with all existent setup.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: upgrade ubuntu from trusty to bionic [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 closed pull request #18286: MINOR: upgrade ubuntu from trusty to bionic
URL: https://github.com/apache/kafka/pull/18286
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: upgrade ubuntu from trusty to bionic [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #18286:
URL: https://github.com/apache/kafka/pull/18286#issuecomment-2560499495
@brandboat do you have free cycle to file a patch? I'd like to keep this PR as it is.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: upgrade ubuntu from trusty to bionic [kafka]
Posted by "jolshan (via GitHub)" <gi...@apache.org>.
jolshan commented on PR #18286:
URL: https://github.com/apache/kafka/pull/18286#issuecomment-2561305255
Thanks @brandboat. I will take a look at that one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: print the error log if we can't list pids [kafka]
Posted by "jolshan (via GitHub)" <gi...@apache.org>.
jolshan commented on PR #18286:
URL: https://github.com/apache/kafka/pull/18286#issuecomment-2557839529
run is here: https://confluent-open-source-kafka-branch-builder-system-test-results.s3-us-west-2.amazonaws.com/trunk/2024-12-20--001.85d9ae90-e619-4689-a395-6d4075645b53--1734723292--chia7712--add_log_pid--acd9853691/report.html
Only the first had the pid issue this time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: print the error log if we can't list pids [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #18286:
URL: https://github.com/apache/kafka/pull/18286#issuecomment-2559883083
> Turns out this also affect committers if their membership visibility in the ASF GitHub org is not public.
done!!!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: upgrade ubuntu from trusty to bionic [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #18286:
URL: https://github.com/apache/kafka/pull/18286#issuecomment-2559981240
@jolshan Could you please test this patch? I am using Ubuntu 18.04 instead of 14.04, and it appears that the `ps ux` command output is no longer truncated.
```
vagrant@worker1:~$ ps ax | grep -i kafka.Kafka
8473 ? Sl 0:02 java -Xmx1G -Xms1G -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+ExplicitGCInvokesConcurrent -XX:MaxInlineLevel=15 -Djava.awt.headless=true -Xlog:gc*:file=/opt/kafka-3.1.2/bin/../logs/kafkaServer-gc.log:time,tags:filecount=10,filesize=100M -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.port=9192 -Dkafka.logs.dir=/opt/kafka-3.1.2/bin/../logs -Dlog4j.configuration=file:/mnt/kafka/log4j.properties -cp /opt/kafka-3.1.2/bin/../libs/activation-1.1.1.jar:/opt/kafka-3.1.2/bin/../libs/aopalliance-repackaged-2.6.1.jar:/opt/kafka-3.1.2/bin/../libs/argparse4j-0.7.0.jar:/opt/kafka-3.1.2/bin/../libs/audience-annotations-0.5.0.jar:/opt/kafka-3.1.2/bin/../libs/commons-cli-1.4.jar:/opt/kafka-3.1.2/bin/../libs/commons-lang3-3.8.1.jar:/opt/kafka-3.1.2/bin/../libs/connect-api-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/connect-basic-
auth-extension-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/connect-json-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/connect-mirror-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/connect-mirror-client-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/connect-runtime-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/connect-transforms-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/hk2-api-2.6.1.jar:/opt/kafka-3.1.2/bin/../libs/hk2-locator-2.6.1.jar:/opt/kafka-3.1.2/bin/../libs/hk2-utils-2.6.1.jar:/opt/kafka-3.1.2/bin/../libs/jackson-annotations-2.12.6.jar:/opt/kafka-3.1.2/bin/../libs/jackson-core-2.12.6.jar:/opt/kafka-3.1.2/bin/../libs/jackson-databind-2.12.6.1.jar:/opt/kafka-3.1.2/bin/../libs/jackson-dataformat-csv-2.12.6.jar:/opt/kafka-3.1.2/bin/../libs/jackson-datatype-jdk8-2.12.6.jar:/opt/kafka-3.1.2/bin/../libs/jackson-jaxrs-base-2.12.6.jar:/opt/kafka-3.1.2/bin/../libs/jackson-jaxrs-json-provider-2.12.6.jar:/opt/kafka-3.1.2/bin/../libs/jackson-module-jaxb-annotations-2.12.6.jar:/opt/kafka-3.1.2/bin/../libs/jackson-module-scala
_2.12-2.12.6.jar:/opt/kafka-3.1.2/bin/../libs/jakarta.activation-api-1.2.1.jar:/opt/kafka-3.1.2/bin/../libs/jakarta.annotation-api-1.3.5.jar:/opt/kafka-3.1.2/bin/../libs/jakarta.inject-2.6.1.jar:/opt/kafka-3.1.2/bin/../libs/jakarta.validation-api-2.0.2.jar:/opt/kafka-3.1.2/bin/../libs/jakarta.ws.rs-api-2.1.6.jar:/opt/kafka-3.1.2/bin/../libs/jakarta.xml.bind-api-2.3.2.jar:/opt/kafka-3.1.2/bin/../libs/javassist-3.27.0-GA.jar:/opt/kafka-3.1.2/bin/../libs/javax.servlet-api-3.1.0.jar:/opt/kafka-3.1.2/bin/../libs/javax.ws.rs-api-2.1.1.jar:/opt/kafka-3.1.2/bin/../libs/jaxb-api-2.3.0.jar:/opt/kafka-3.1.2/bin/../libs/jersey-client-2.34.jar:/opt/kafka-3.1.2/bin/../libs/jersey-common-2.34.jar:/opt/kafka-3.1.2/bin/../libs/jersey-container-servlet-2.34.jar:/opt/kafka-3.1.2/bin/../libs/jersey-container-servlet-core-2.34.jar:/opt/kafka-3.1.2/bin/../libs/jersey-hk2-2.34.jar:/opt/kafka-3.1.2/bin/../libs/jersey-server-2.34.jar:/opt/kafka-3.1.2/bin/../libs/jetty-client-9.4.48.v20220622.jar:/opt/kafka-
3.1.2/bin/../libs/jetty-continuation-9.4.48.v20220622.jar:/opt/kafka-3.1.2/bin/../libs/jetty-http-9.4.48.v20220622.jar:/opt/kafka-3.1.2/bin/../libs/jetty-io-9.4.48.v20220622.jar:/opt/kafka-3.1.2/bin/../libs/jetty-security-9.4.48.v20220622.jar:/opt/kafka-3.1.2/bin/../libs/jetty-server-9.4.48.v20220622.jar:/opt/kafka-3.1.2/bin/../libs/jetty-servlet-9.4.48.v20220622.jar:/opt/kafka-3.1.2/bin/../libs/jetty-servlets-9.4.48.v20220622.jar:/opt/kafka-3.1.2/bin/../libs/jetty-util-9.4.48.v20220622.jar:/opt/kafka-3.1.2/bin/../libs/jetty-util-ajax-9.4.48.v20220622.jar:/opt/kafka-3.1.2/bin/../libs/jline-3.12.1.jar:/opt/kafka-3.1.2/bin/../libs/jopt-simple-5.0.4.jar:/opt/kafka-3.1.2/bin/../libs/jose4j-0.7.8.jar:/opt/kafka-3.1.2/bin/../libs/kafka-clients-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/kafka-log4j-appender-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/kafka-metadata-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/kafka-raft-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/kafka-server-common-3.1.2.jar:/opt/kafka-3.1.2
/bin/../libs/kafka-shell-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/kafka-storage-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/kafka-storage-api-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/kafka-streams-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/kafka-streams-examples-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/kafka-streams-scala_2.12-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/kafka-streams-test-utils-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/kafka-tools-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/kafka_2.12-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/lz4-java-1.8.0.jar:/opt/kafka-3.1.2/bin/../libs/maven-artifact-3.8.1.jar:/opt/kafka-3.1.2/bin/../libs/metrics-core-2.2.0.jar:/opt/kafka-3.1.2/bin/../libs/metrics-core-4.1.12.1.jar:/opt/kafka-3.1.2/bin/../libs/netty-buffer-4.1.73.Final.jar:/opt/kafka-3.1.2/bin/../libs/netty-codec-4.1.73.Final.jar:/opt/kafka-3.1.2/bin/../libs/netty-common-4.1.73.Final.jar:/opt/kafka-3.1.2/bin/../libs/netty-handler-4.1.73.Final.jar:/opt/kafka-3.1.2/bin/../libs/netty-resolver-4.1.73.Final.jar:/o
pt/kafka-3.1.2/bin/../libs/netty-tcnative-classes-2.0.46.Final.jar:/opt/kafka-3.1.2/bin/../libs/netty-transport-4.1.73.Final.jar:/opt/kafka-3.1.2/bin/../libs/netty-transport-classes-epoll-4.1.73.Final.jar:/opt/kafka-3.1.2/bin/../libs/netty-transport-native-epoll-4.1.73.Final.jar:/opt/kafka-3.1.2/bin/../libs/netty-transport-native-unix-common-4.1.73.Final.jar:/opt/kafka-3.1.2/bin/../libs/osgi-resource-locator-1.0.3.jar:/opt/kafka-3.1.2/bin/../libs/paranamer-2.8.jar:/opt/kafka-3.1.2/bin/../libs/plexus-utils-3.2.1.jar:/opt/kafka-3.1.2/bin/../libs/reflections-0.9.12.jar:/opt/kafka-3.1.2/bin/../libs/reload4j-1.2.19.jar:/opt/kafka-3.1.2/bin/../libs/rocksdbjni-6.22.1.1.jar:/opt/kafka-3.1.2/bin/../libs/scala-collection-compat_2.12-2.4.4.jar:/opt/kafka-3.1.2/bin/../libs/scala-java8-compat_2.12-1.0.0.jar:/opt/kafka-3.1.2/bin/../libs/scala-library-2.12.14.jar:/opt/kafka-3.1.2/bin/../libs/scala-logging_2.12-3.9.3.jar:/opt/kafka-3.1.2/bin/../libs/scala-reflect-2.12.14.jar:/opt/kafka-3.1.2/bin/..
/libs/slf4j-api-1.7.36.jar:/opt/kafka-3.1.2/bin/../libs/slf4j-reload4j-1.7.36.jar:/opt/kafka-3.1.2/bin/../libs/snappy-java-1.1.8.4.jar:/opt/kafka-3.1.2/bin/../libs/trogdor-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/zookeeper-3.6.3.jar:/opt/kafka-3.1.2/bin/../libs/zookeeper-jute-3.6.3.jar:/opt/kafka-3.1.2/bin/../libs/zstd-jni-1.5.0-4.jar -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/mnt/kafka/kafka_heap_dump.bin kafka.Kafka /mnt/kafka/kafka.properties
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: upgrade ubuntu from trusty to bionic [kafka]
Posted by "jolshan (via GitHub)" <gi...@apache.org>.
jolshan commented on PR #18286:
URL: https://github.com/apache/kafka/pull/18286#issuecomment-2560366700
This was my old branch which had the ntpupdate in a slightly different place (it also didn't work for me at the time though)
https://github.com/apache/kafka/compare/trunk...jolshan:kafka:ami-update
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: upgrade ubuntu from trusty to bionic [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #18286:
URL: https://github.com/apache/kafka/pull/18286#issuecomment-2561521149
@brandboat should we fix all other cases using "ps ux" to avoid similar issue in the future?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: print the error log if we can't list pids [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #18286:
URL: https://github.com/apache/kafka/pull/18286#issuecomment-2557975232
> Is this only to debug the issue or is it generally useful?
in fact, this PR is a no-op due to `allow_fail=True` - ducktape uses log instead of exception :_
https://github.com/confluentinc/ducktape/blob/master/ducktape/cluster/remoteaccount.py#L366C51-L366C70
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: print the error log if we can't list pids [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #18286:
URL: https://github.com/apache/kafka/pull/18286#issuecomment-2558428262
@jolshan the root cause is that the output of `ps ax` is truncated. see following output
```
5150 ? Sl 0:04 java -Xmx1G -Xms1G -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+ExplicitGCInvokesConcurrent -XX:MaxInlineLevel=15 -Djava.awt.headless=true -Xlog:gc*:file=/opt/kafka-3.1.2/bin/../logs/kafkaServer-gc.log:time,tags:filecount=10,filesize=100M -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.port=9192 -Dkafka.logs.dir=/opt/kafka-3.1.2/bin/../logs -Dlog4j.configuration=file:/mnt/kafka/log4j.properties -cp /opt/kafka-3.1.2/bin/../libs/activation-1.1.1.jar:/opt/kafka-3.1.2/bin/../libs/aopalliance-repackaged-2.6.1.jar:/opt/kafka-3.1.2/bin/../libs/argparse4j-0.7.0.jar:/opt/kafka-3.1.2/bin/../libs/audience-annotations-0.5.0.jar:/opt/kafka-3.1.2/bin/../libs/commons-cli-1.4.jar:/opt/kafka-3.1.2/bin/../libs/commons-lang3-3.8.1.jar:/opt/kafka-3.1.2/bin/../libs/connect-api-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/connect-basic-
auth-extension-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/connect-json-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/connect-mirror-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/connect-mirror-client-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/connect-runtime-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/connect-transforms-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/hk2-api-2.6.1.jar:/opt/kafka-3.1.2/bin/../libs/hk2-locator-2.6.1.jar:/opt/kafka-3.1.2/bin/../libs/hk2-utils-2.6.1.jar:/opt/kafka-3.1.2/bin/../libs/jackson-annotations-2.12.6.jar:/opt/kafka-3.1.2/bin/../libs/jackson-core-2.12.6.jar:/opt/kafka-3.1.2/bin/../libs/jackson-databind-2.12.6.1.jar:/opt/kafka-3.1.2/bin/../libs/jackson-dataformat-csv-2.12.6.jar:/opt/kafka-3.1.2/bin/../libs/jackson-datatype-jdk8-2.12.6.jar:/opt/kafka-3.1.2/bin/../libs/jackson-jaxrs-base-2.12.6.jar:/opt/kafka-3.1.2/bin/../libs/jackson-jaxrs-json-provider-2.12.6.jar:/opt/kafka-3.1.2/bin/../libs/jackson-module-jaxb-annotations-2.12.6.jar:/opt/kafka-3.1.2/bin/../libs/jackson-module-scala
_2.12-2.12.6.jar:/opt/kafka-3.1.2/bin/../libs/jakarta.activation-api-1.2.1.jar:/opt/kafka-3.1.2/bin/../libs/jakarta.annotation-api-1.3.5.jar:/opt/kafka-3.1.2/bin/../libs/jakarta.inject-2.6.1.jar:/opt/kafka-3.1.2/bin/../libs/jakarta.validation-api-2.0.2.jar:/opt/kafka-3.1.2/bin/../libs/jakarta.ws.rs-api-2.1.6.jar:/opt/kafka-3.1.2/bin/../libs/jakarta.xml.bind-api-2.3.2.jar:/opt/kafka-3.1.2/bin/../libs/javassist-3.27.0-GA.jar:/opt/kafka-3.1.2/bin/../libs/javax.servlet-api-3.1.0.jar:/opt/kafka-3.1.2/bin/../libs/javax.ws.rs-api-2.1.1.jar:/opt/kafka-3.1.2/bin/../libs/jaxb-api-2.3.0.jar:/opt/kafka-3.1.2/bin/../libs/jersey-client-2.34.jar:/opt/kafka-3.1.2/bin/../libs/jersey-common-2.34.jar:/opt/kafka-3.1.2/bin/../libs/jersey-container-servlet-2.34.jar:/opt/kafka-3.1.2/bin/../libs/jersey-container-servlet-core-2.34.jar:/opt/kafka-3.1.2/bin/../libs/jersey-hk2-2.34.jar:/opt/kafka-3.1.2/bin/../libs/jersey-server-2.34.jar:/opt/kafka-3.1.2/bin/../libs/jetty-client-9.4.48.v20220622.jar:/opt/kafka-
3.1.2/bin/../libs/jetty-continuation-9.4.48.v20220622.jar:/opt/kafka-3.1.2/bin/../libs/jetty-http-9.4.48.v20220622.jar:/opt/kafka-3.1.2/bin/../libs/jetty-io-9.4.48.v20220622.jar:/opt/kafka-3.1.2/bin/../libs/jetty-security-9.4.48.v20220622.jar:/opt/kafka-3.1.2/bin/../libs/jetty-server-9.4.48.v20220622.jar:/opt/kafka-3.1.2/bin/../libs/jetty-servlet-9.4.48.v20220622.jar:/opt/kafka-3.1.2/bin/../libs/jetty-servlets-9.4.48.v20220622.jar:/opt/kafka-3.1.2/bin/../libs/jetty-util-9.4.48.v20220622.jar:/opt/kafka-3.1.2/bin/../libs/jetty-util-ajax-9.4.48.v20220622.jar:/opt/kafka-3.1.2/bin/../libs/jline-3.12.1.jar:/opt/kafka-3.1.2/bin/../libs/jopt-simple-5.0.4.jar:/opt/kafka-3.1.2/bin/../libs/jose4j-0.7.8.jar:/opt/kafka-3.1.2/bin/../libs/kafka_2.12-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/kafka-clients-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/kafka-log4j-appender-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/kafka-metadata-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/kafka-raft-3.1.2.jar:/opt/kafka-3.1.2/bin/../l
ibs/kafka-server-common-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/kafka-shell-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/kafka-storage-3.1.2.jar
```
there is no keyword "kafka.Kafka" as it is truncated.
the following output is from container
```
534 ? Sl 0:09 java -Xmx1G -Xms1G -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+ExplicitGCInvokesConcurrent -XX:MaxInlineLevel=15 -Djava.awt.headless=true -Xlog:gc*:file=/opt/kafka-3.1.2/bin/../logs/kafkaServer-gc.log:time,tags:filecount=10,filesize=100M -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.port=9192 -Dkafka.logs.dir=/opt/kafka-3.1.2/bin/../logs -Dlog4j.configuration=file:/mnt/kafka/log4j.properties -cp /opt/kafka-3.1.2/bin/../libs/activation-1.1.1.jar:/opt/kafka-3.1.2/bin/../libs/aopalliance-repackaged-2.6.1.jar:/opt/kafka-3.1.2/bin/../libs/argparse4j-0.7.0.jar:/opt/kafka-3.1.2/bin/../libs/audience-annotations-0.5.0.jar:/opt/kafka-3.1.2/bin/../libs/commons-cli-1.4.jar:/opt/kafka-3.1.2/bin/../libs/commons-lang3-3.8.1.jar:/opt/kafka-3.1.2/bin/../libs/connect-api-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/connect-basi
c-auth-extension-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/connect-json-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/connect-mirror-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/connect-mirror-client-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/connect-runtime-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/connect-transforms-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/hk2-api-2.6.1.jar:/opt/kafka-3.1.2/bin/../libs/hk2-locator-2.6.1.jar:/opt/kafka-3.1.2/bin/../libs/hk2-utils-2.6.1.jar:/opt/kafka-3.1.2/bin/../libs/jackson-annotations-2.12.6.jar:/opt/kafka-3.1.2/bin/../libs/jackson-core-2.12.6.jar:/opt/kafka-3.1.2/bin/../libs/jackson-databind-2.12.6.1.jar:/opt/kafka-3.1.2/bin/../libs/jackson-dataformat-csv-2.12.6.jar:/opt/kafka-3.1.2/bin/../libs/jackson-datatype-jdk8-2.12.6.jar:/opt/kafka-3.1.2/bin/../libs/jackson-jaxrs-base-2.12.6.jar:/opt/kafka-3.1.2/bin/../libs/jackson-jaxrs-json-provider-2.12.6.jar:/opt/kafka-3.1.2/bin/../libs/jackson-module-jaxb-annotations-2.12.6.jar:/opt/kafka-3.1.2/bin/../libs/jackson-module-sca
la_2.12-2.12.6.jar:/opt/kafka-3.1.2/bin/../libs/jakarta.activation-api-1.2.1.jar:/opt/kafka-3.1.2/bin/../libs/jakarta.annotation-api-1.3.5.jar:/opt/kafka-3.1.2/bin/../libs/jakarta.inject-2.6.1.jar:/opt/kafka-3.1.2/bin/../libs/jakarta.validation-api-2.0.2.jar:/opt/kafka-3.1.2/bin/../libs/jakarta.ws.rs-api-2.1.6.jar:/opt/kafka-3.1.2/bin/../libs/jakarta.xml.bind-api-2.3.2.jar:/opt/kafka-3.1.2/bin/../libs/javassist-3.27.0-GA.jar:/opt/kafka-3.1.2/bin/../libs/javax.servlet-api-3.1.0.jar:/opt/kafka-3.1.2/bin/../libs/javax.ws.rs-api-2.1.1.jar:/opt/kafka-3.1.2/bin/../libs/jaxb-api-2.3.0.jar:/opt/kafka-3.1.2/bin/../libs/jersey-client-2.34.jar:/opt/kafka-3.1.2/bin/../libs/jersey-common-2.34.jar:/opt/kafka-3.1.2/bin/../libs/jersey-container-servlet-2.34.jar:/opt/kafka-3.1.2/bin/../libs/jersey-container-servlet-core-2.34.jar:/opt/kafka-3.1.2/bin/../libs/jersey-hk2-2.34.jar:/opt/kafka-3.1.2/bin/../libs/jersey-server-2.34.jar:/opt/kafka-3.1.2/bin/../libs/jetty-client-9.4.48.v20220622.jar:/opt/kafk
a-3.1.2/bin/../libs/jetty-continuation-9.4.48.v20220622.jar:/opt/kafka-3.1.2/bin/../libs/jetty-http-9.4.48.v20220622.jar:/opt/kafka-3.1.2/bin/../libs/jetty-io-9.4.48.v20220622.jar:/opt/kafka-3.1.2/bin/../libs/jetty-security-9.4.48.v20220622.jar:/opt/kafka-3.1.2/bin/../libs/jetty-server-9.4.48.v20220622.jar:/opt/kafka-3.1.2/bin/../libs/jetty-servlet-9.4.48.v20220622.jar:/opt/kafka-3.1.2/bin/../libs/jetty-servlets-9.4.48.v20220622.jar:/opt/kafka-3.1.2/bin/../libs/jetty-util-9.4.48.v20220622.jar:/opt/kafka-3.1.2/bin/../libs/jetty-util-ajax-9.4.48.v20220622.jar:/opt/kafka-3.1.2/bin/../libs/jline-3.12.1.jar:/opt/kafka-3.1.2/bin/../libs/jopt-simple-5.0.4.jar:/opt/kafka-3.1.2/bin/../libs/jose4j-0.7.8.jar:/opt/kafka-3.1.2/bin/../libs/kafka-clients-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/kafka-log4j-appender-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/kafka-metadata-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/kafka-raft-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/kafka-server-common-3.1.2.jar:/opt/kafka-3.1
.2/bin/../libs/kafka-shell-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/kafka-storage-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/kafka-storage-api-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/kafka-streams-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/kafka-streams-examples-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/kafka-streams-scala_2.12-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/kafka-streams-test-utils-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/kafka-tools-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/kafka_2.12-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/lz4-java-1.8.0.jar:/opt/kafka-3.1.2/bin/../libs/maven-artifact-3.8.1.jar:/opt/kafka-3.1.2/bin/../libs/metrics-core-2.2.0.jar:/opt/kafka-3.1.2/bin/../libs/metrics-core-4.1.12.1.jar:/opt/kafka-3.1.2/bin/../libs/netty-buffer-4.1.73.Final.jar:/opt/kafka-3.1.2/bin/../libs/netty-codec-4.1.73.Final.jar:/opt/kafka-3.1.2/bin/../libs/netty-common-4.1.73.Final.jar:/opt/kafka-3.1.2/bin/../libs/netty-handler-4.1.73.Final.jar:/opt/kafka-3.1.2/bin/../libs/netty-resolver-4.1.73.Final.jar:
/opt/kafka-3.1.2/bin/../libs/netty-tcnative-classes-2.0.46.Final.jar:/opt/kafka-3.1.2/bin/../libs/netty-transport-4.1.73.Final.jar:/opt/kafka-3.1.2/bin/../libs/netty-transport-classes-epoll-4.1.73.Final.jar:/opt/kafka-3.1.2/bin/../libs/netty-transport-native-epoll-4.1.73.Final.jar:/opt/kafka-3.1.2/bin/../libs/netty-transport-native-unix-common-4.1.73.Final.jar:/opt/kafka-3.1.2/bin/../libs/osgi-resource-locator-1.0.3.jar:/opt/kafka-3.1.2/bin/../libs/paranamer-2.8.jar:/opt/kafka-3.1.2/bin/../libs/plexus-utils-3.2.1.jar:/opt/kafka-3.1.2/bin/../libs/reflections-0.9.12.jar:/opt/kafka-3.1.2/bin/../libs/reload4j-1.2.19.jar:/opt/kafka-3.1.2/bin/../libs/rocksdbjni-6.22.1.1.jar:/opt/kafka-3.1.2/bin/../libs/scala-collection-compat_2.12-2.4.4.jar:/opt/kafka-3.1.2/bin/../libs/scala-java8-compat_2.12-1.0.0.jar:/opt/kafka-3.1.2/bin/../libs/scala-library-2.12.14.jar:/opt/kafka-3.1.2/bin/../libs/scala-logging_2.12-3.9.3.jar:/opt/kafka-3.1.2/bin/../libs/scala-reflect-2.12.14.jar:/opt/kafka-3.1.2/bin/
../libs/slf4j-api-1.7.36.jar:/opt/kafka-3.1.2/bin/../libs/slf4j-reload4j-1.7.36.jar:/opt/kafka-3.1.2/bin/../libs/snappy-java-1.1.8.4.jar:/opt/kafka-3.1.2/bin/../libs/trogdor-3.1.2.jar:/opt/kafka-3.1.2/bin/../libs/zookeeper-3.6.3.jar:/opt/kafka-3.1.2/bin/../libs/zookeeper-jute-3.6.3.jar:/opt/kafka-3.1.2/bin/../libs/zstd-jni-1.5.0-4.jar -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/mnt/kafka/kafka_heap_dump.bin kafka.Kafka /mnt/kafka/kafka.properties
```
Log4j2 introduces new jars to the classpath, which leads to truncation issues.
I have some preliminary approaches to address this problem and will update this PR later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: print the error log if we can't list pids [kafka]
Posted by "mumrah (via GitHub)" <gi...@apache.org>.
mumrah commented on PR #18286:
URL: https://github.com/apache/kafka/pull/18286#issuecomment-2559848332
@chia7712 https://github.com/apache/kafka/pull/17881 adds a "triage" label to PRs from non-committers. Turns out this also affect committers if their membership visibility in the ASF GitHub org is not public. I added instructions for setting your membership visibility to public https://github.com/apache/kafka/blob/trunk/.github/workflows/README.md#pr-triage
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: upgrade ubuntu from trusty to bionic [kafka]
Posted by "brandboat (via GitHub)" <gi...@apache.org>.
brandboat commented on PR #18286:
URL: https://github.com/apache/kafka/pull/18286#issuecomment-2561264584
gentle ping @jolshan, I just filed https://github.com/apache/kafka/pull/18312 which use `jcmd` instead of `ps ax` to mitigate the output truncate issue, perhaps you can give it a try, thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: upgrade ubuntu from trusty to bionic [kafka]
Posted by "brandboat (via GitHub)" <gi...@apache.org>.
brandboat commented on PR #18286:
URL: https://github.com/apache/kafka/pull/18286#issuecomment-2561544840
> @brandboat should we fix all other cases using "ps ux" to avoid similar issue in the future?
There is another ps us usage in https://github.com/apache/kafka/blob/4ed7f2cd01deabca5cf81383d8c9ee68609f8b45/tests/kafkatest/utils/util.py#L60. What's interesting about this one is that it parses the Kafka jar in the classpath to retrieve the version. However, if we switch to using java_pids, this functionality would no longer be possible. I'm afraid that we have to leave it as it is.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org