You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "maniekes (via GitHub)" <gi...@apache.org> on 2023/06/15 13:54:32 UTC
[GitHub] [kafka] maniekes opened a new pull request, #13321: MINOR: added compatibility for MinGW
maniekes opened a new pull request, #13321:
URL: https://github.com/apache/kafka/pull/13321
kafka class runner is not working from MINGW/Git Bash on Windows:
```bash
rafal@rafal-laptok MINGW64 /c/rafal/git/configs (master)
$ "$KAFKA_DIR/bin/kafka-server-start.sh" server.properties
[0.004s][error][logging] Error opening log file '/c/rafal/tools/kafka_2.13-3.3.2/bin/../logs/kafkaServer-gc.log': No such file or directory
[0.004s][error][logging] Initialization of output 'file=/c/rafal/tools/kafka_2.13-3.3.2/bin/../logs/kafkaServer-gc.log' using options 'filecount=10,filesize=100M' failed.
Invalid -Xlog option '-Xlog:gc*:file=/c/rafal/tools/kafka_2.13-3.3.2/bin/../logs/kafkaServer-gc.log:time,tags:filecount=10,filesize=100M', see error log for details.
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
```
however it works fine when using Cygwin. what i can see it has already logic for changing directory separators, so to fix this issue I just added extra condition for MINGW. it fixed issue on my pc.
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: added compatibility for MinGW [kafka]
Posted by "divijvaidya (via GitHub)" <gi...@apache.org>.
divijvaidya commented on code in PR #13321:
URL: https://github.com/apache/kafka/pull/13321#discussion_r1372856100
##########
bin/kafka-run-class.sh:
##########
@@ -229,7 +230,7 @@ if [ -z "$KAFKA_LOG4J_OPTS" ]; then
# Log to console. This is a tool.
LOG4J_DIR="$base_dir/config/tools-log4j.properties"
# If Cygwin is detected, LOG4J_DIR is converted to Windows format.
- (( CYGWIN )) && LOG4J_DIR=$(cygpath --path --mixed "${LOG4J_DIR}")
+ (( WINDOWS_OS_FORMAT )) && LOG4J_DIR=$(cygpath --path --mixed "${LOG4J_DIR}")
KAFKA_LOG4J_OPTS="-Dlog4j.configuration=file:${LOG4J_DIR}"
Review Comment:
gotcha! I had missed that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] divijvaidya commented on pull request #13321: MINOR: added compatibility for MinGW
Posted by "divijvaidya (via GitHub)" <gi...@apache.org>.
divijvaidya commented on PR #13321:
URL: https://github.com/apache/kafka/pull/13321#issuecomment-1594696603
> hope you test is also in pipeline for cygwin to be double sure
No, we don't have any tests based on windows (or cygwin) in our pipeline. We don't officially promise to support Kafka on windows platform. No pressure but we are relying on your testing here :)
> what i noticed that msys2 was not reading paths properly because path translator for cygwin was adding backslashes instead of slashes
msys2 uses the same `cygpath` as cygwin so why isn't it working for msys2. Do you have any idea? My concern with using `--windows` instead of `--mixed` is that this would be a breaking change for people who have written scripts around this bug/feature of using the `/` slash. Hence, ideally I think it would be preferable to keep the existing behaviour as same.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: added compatibility for MinGW [kafka]
Posted by "divijvaidya (via GitHub)" <gi...@apache.org>.
divijvaidya commented on code in PR #13321:
URL: https://github.com/apache/kafka/pull/13321#discussion_r1371421492
##########
bin/kafka-run-class.sh:
##########
@@ -229,7 +230,7 @@ if [ -z "$KAFKA_LOG4J_OPTS" ]; then
# Log to console. This is a tool.
LOG4J_DIR="$base_dir/config/tools-log4j.properties"
# If Cygwin is detected, LOG4J_DIR is converted to Windows format.
- (( CYGWIN )) && LOG4J_DIR=$(cygpath --path --mixed "${LOG4J_DIR}")
+ (( WINDOWS_OS_FORMAT )) && LOG4J_DIR=$(cygpath --path --mixed "${LOG4J_DIR}")
KAFKA_LOG4J_OPTS="-Dlog4j.configuration=file:${LOG4J_DIR}"
Review Comment:
We are not using `KAFKA_LOG4J_OPTS` this at the end of the script now, hence this logic will not be executed.
Could we please revert back to using `KAFKA_LOG4J_OPTS`? Also asking because folks may already have scripts where they set `KAFKA_LOG4J_OPTS` as an environment variable. We don't want to break that, hence we need to continue using `KAFKA_LOG4J_OPTS`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15685: added compatibility for MinGW [kafka]
Posted by "maniekes (via GitHub)" <gi...@apache.org>.
maniekes commented on PR #13321:
URL: https://github.com/apache/kafka/pull/13321#issuecomment-1780816193
thank you @divijvaidya for help and support. i created jira account in Jira but cannot assign ticket to me, can you try to assign now?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] maniekes commented on pull request #13321: MINOR: added compatibility for MinGW
Posted by "maniekes (via GitHub)" <gi...@apache.org>.
maniekes commented on PR #13321:
URL: https://github.com/apache/kafka/pull/13321#issuecomment-1646799859
hi @divijvaidya what do u think about my last changes?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] maniekes commented on pull request #13321: MINOR: added compatibility for MinGW
Posted by "maniekes (via GitHub)" <gi...@apache.org>.
maniekes commented on PR #13321:
URL: https://github.com/apache/kafka/pull/13321#issuecomment-1585694903
hi @divijvaidya @mimaison @ijuma this PR got marked as stale, what do you think about merging it or closing?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15685: added compatibility for MinGW [kafka]
Posted by "mimaison (via GitHub)" <gi...@apache.org>.
mimaison commented on PR #13321:
URL: https://github.com/apache/kafka/pull/13321#issuecomment-2058597443
To run the `GetOffsetShell` tool you should use the dedicated script `bin/kafka-get-offsets.sh`.
The error you get is because the class was moved, it's now `org.apache.kafka.tools.GetOffsetShell`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15685: added compatibility for MinGW [kafka]
Posted by "divijvaidya (via GitHub)" <gi...@apache.org>.
divijvaidya commented on PR #13321:
URL: https://github.com/apache/kafka/pull/13321#issuecomment-1780956839
What is your Jira ID / Full name? Also to double check, you created an account in Apache Jira, right? The instructions are at https://selfserve.apache.org/jira-account.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] divijvaidya commented on pull request #13321: MINOR: added compatibility for MinGW
Posted by "divijvaidya (via GitHub)" <gi...@apache.org>.
divijvaidya commented on PR #13321:
URL: https://github.com/apache/kafka/pull/13321#issuecomment-1449651799
Hi @maniekes, I see that this is your first contribution to Apache Kafka. Congratulations :tada:
However, I have some reservations about this change. We try to ensure that whatever we ship in a version release works fine. For that, we perform varies types of unit, integration and version upgrade tests during the release process. These are executed via jenkins on Apache infrastructure hosts.
But changes such as this PR can not be tested since we do not have testing infra for it. Which means, over a period of time, it would not be possible to support Mingw64 . Now, we could either say, that as a project we will do a best effort support for mingw64 but can't guarantee it will work (since we don't test it) OR we can say that we don't support it at all. The latter approach is not inclusive but the former is flaky at best.
I am tagging some PMC members here to hear about what they have to say about adding code for non-standard/unsupported platforms in our code base. Let's go from there.
@mimaison @ijuma what are your thoughts on the topic on testability and addition of support for various platform in our code base? Do we require a DISCUSS thread in mailing list to discuss this further or do we merge this since it is a small change in the bash scripts?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] github-actions[bot] commented on pull request #13321: MINOR: added compatibility for MinGW
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13321:
URL: https://github.com/apache/kafka/pull/13321#issuecomment-1585687212
This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has merge conflicts, please update it with the latest from trunk (or appropriate release branch)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] divijvaidya commented on pull request #13321: MINOR: added compatibility for MinGW
Posted by "divijvaidya (via GitHub)" <gi...@apache.org>.
divijvaidya commented on PR #13321:
URL: https://github.com/apache/kafka/pull/13321#issuecomment-1594389639
Thank you @maniekes. MINGW64 looks good. Can you also please provide test details for MSYS?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15685: added compatibility for MinGW [kafka]
Posted by "maniekes (via GitHub)" <gi...@apache.org>.
maniekes commented on PR #13321:
URL: https://github.com/apache/kafka/pull/13321#issuecomment-1780967748
yes, i created one. maniekes / Rafal Klimonda. i left comment under this issue so you can easly idenfity me
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: added compatibility for MinGW [kafka]
Posted by "maniekes (via GitHub)" <gi...@apache.org>.
maniekes commented on code in PR #13321:
URL: https://github.com/apache/kafka/pull/13321#discussion_r1371452345
##########
bin/kafka-run-class.sh:
##########
@@ -229,7 +230,7 @@ if [ -z "$KAFKA_LOG4J_OPTS" ]; then
# Log to console. This is a tool.
LOG4J_DIR="$base_dir/config/tools-log4j.properties"
# If Cygwin is detected, LOG4J_DIR is converted to Windows format.
- (( CYGWIN )) && LOG4J_DIR=$(cygpath --path --mixed "${LOG4J_DIR}")
+ (( WINDOWS_OS_FORMAT )) && LOG4J_DIR=$(cygpath --path --mixed "${LOG4J_DIR}")
KAFKA_LOG4J_OPTS="-Dlog4j.configuration=file:${LOG4J_DIR}"
Review Comment:
i did not modified this line, it was from the beginning. if you expand view to line 228 you will see that this code will execute only if `KAFKA_LOG4J_OPTS` is not set so it will apply defaults to it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] divijvaidya commented on pull request #13321: MINOR: added compatibility for MinGW
Posted by "divijvaidya (via GitHub)" <gi...@apache.org>.
divijvaidya commented on PR #13321:
URL: https://github.com/apache/kafka/pull/13321#issuecomment-1593219444
@maniekes can you please describe the testing steps and the results of the tests that you did to test the latest version of this change?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: added compatibility for MinGW [kafka]
Posted by "maniekes (via GitHub)" <gi...@apache.org>.
maniekes commented on PR #13321:
URL: https://github.com/apache/kafka/pull/13321#issuecomment-1777659808
@divijvaidya looks like it's stale PR :) shall we merge it or close?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15685: added compatibility for MinGW [kafka]
Posted by "divijvaidya (via GitHub)" <gi...@apache.org>.
divijvaidya merged PR #13321:
URL: https://github.com/apache/kafka/pull/13321
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] maniekes commented on pull request #13321: MINOR: added compatibility for MinGW
Posted by "maniekes (via GitHub)" <gi...@apache.org>.
maniekes commented on PR #13321:
URL: https://github.com/apache/kafka/pull/13321#issuecomment-1593543359
@divijvaidya, sure, it's quite simple.
1. open git bash, checkout trunk
2. build
```
./gradlew jar -PscalaVersion=2.13.10
```
3. checkout trunk before my commits:
`git checkout HEAD~~`
4. start zookeeper:
```
****@rafal-laptok MINGW64 /c/rafal/git/maniekes-kafka ((c5df47a1cb...))
$ ./bin/zookeeper-server-start.sh config/zookeeper.properties
[0.003s][error][logging] Error opening log file '/c/rafal/git/maniekes-kafka/bin/../logs/zookeeper-gc.log': No such file or directory
[0.003s][error][logging] Initialization of output 'file=/c/rafal/git/maniekes-kafka/bin/../logs/zookeeper-gc.log' using options 'filecount=10,filesize=100M' failed.
Invalid -Xlog option '-Xlog:gc*:file=/c/rafal/git/maniekes-kafka/bin/../logs/zookeeper-gc.log:time,tags:filecount=10,filesize=100M', see error log for details.
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
```
5. start kafka:
```
****@rafal-laptok MINGW64 /c/rafal/git/maniekes-kafka ((c5df47a1cb...))
$ ./bin/kafka-server-start.sh config/server.properties
[0.003s][error][logging] Error opening log file '/c/rafal/git/maniekes-kafka/bin/../logs/kafkaServer-gc.log': No such file or directory
[0.003s][error][logging] Initialization of output 'file=/c/rafal/git/maniekes-kafka/bin/../logs/kafkaServer-gc.log' using options 'filecount=10,filesize=100M' failed.
Invalid -Xlog option '-Xlog:gc*:file=/c/rafal/git/maniekes-kafka/bin/../logs/kafkaServer-gc.log:time,tags:filecount=10,filesize=100M', see error log for details.
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
```
6. checkout trunk
```
git checkout trunk
```
7. start zookepper:
```
*****@rafal-laptok MINGW64 /c/rafal/git/maniekes-kafka (trunk)
$ ./bin/zookeeper-server-start.sh config/zookeeper.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/tools/build/dependant-libs-2.13.10/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/trogdor/build/dependant-libs-2.13.10/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/connect/runtime/build/dependant-libs/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/connect/mirror/build/dependant-libs/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Reload4jLoggerFactory]
[2023-06-15 20:26:34,480] INFO Reading configuration from: config/zookeeper.properties (org.apache.zookeeper.server.quorum.QuorumPeerConfig)
[2023-06-15 20:26:34,481] WARN config\zookeeper.properties is relative. Prepend .\ to indicate that you're sure! (org.apache.zookeeper.server.quorum.QuorumPeerConfig)
[2023-06-15 20:26:34,482] WARN \tmp\zookeeper is relative. Prepend .\ to indicate that you're sure! (org.apache.zookeeper.server.quorum.QuorumPeerConfig)
[2023-06-15 20:26:34,483] INFO clientPortAddress is 0.0.0.0:2181 (org.apache.zookeeper.server.quorum.QuorumPeerConfig)
[2023-06-15 20:26:34,483] INFO secureClientPort is not set (org.apache.zookeeper.server.quorum.QuorumPeerConfig)
[2023-06-15 20:26:34,483] INFO observerMasterPort is not set (org.apache.zookeeper.server.quorum.QuorumPeerConfig)
[2023-06-15 20:26:34,484] INFO metricsProvider.className is org.apache.zookeeper.metrics.impl.DefaultMetricsProvider (org.apache.zookeeper.server.quorum.QuorumPeerConfig)
[2023-06-15 20:26:34,485] INFO autopurge.snapRetainCount set to 3 (org.apache.zookeeper.server.DatadirCleanupManager)
[2023-06-15 20:26:34,485] INFO autopurge.purgeInterval set to 0 (org.apache.zookeeper.server.DatadirCleanupManager)
..... (all starts fine)
```
8. start kafka:
```
****@rafal-laptok MINGW64 /c/rafal/git/maniekes-kafka (trunk)
$ ./bin/kafka-server-start.sh config/server.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/tools/build/dependant-libs-2.13.10/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/trogdor/build/dependant-libs-2.13.10/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/connect/runtime/build/dependant-libs/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/connect/mirror/build/dependant-libs/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Reload4jLoggerFactory]
[2023-06-15 20:27:22,294] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)
[2023-06-15 20:27:22,451] INFO Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation (org.apache.zookeeper.common.X509Util)
[2023-06-15 20:27:22,512] INFO starting (kafka.server.KafkaServer)
[2023-06-15 20:27:22,513] INFO Connecting to zookeeper on localhost:2181 (kafka.server.KafkaServer)
[2023-06-15 20:27:22,521] INFO [ZooKeeperClient Kafka server] Initializing a new session to localhost:2181. (kafka.zookeeper.ZooKeeperClient)
[2023-06-15 20:27:22,549] INFO Client environment:zookeeper.version=3.6.4--d65253dcf68e9097c6e95a126463fd5fdeb4521c, built on 12/18/2022 18:10 GMT (org.apache.zookeeper.ZooKeeper)
[2023-06-15 20:27:22,550] INFO Client environment:host.name=192.168.1.137 (org.apache.zookeeper.ZooKeeper)
[2023-06-15 20:27:22,550] INFO Client environment:java.version=19.0.1 (org.apache.zookeeper.ZooKeeper)
[2023-06-15 20:27:22,550] INFO Client environment:java.vendor=Oracle Corporation (org.apache.zookeeper.ZooKeeper)
... (all looks fine)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] maniekes commented on pull request #13321: MINOR: added compatibility for MinGW
Posted by "maniekes (via GitHub)" <gi...@apache.org>.
maniekes commented on PR #13321:
URL: https://github.com/apache/kafka/pull/13321#issuecomment-1594900986
some extra research. i rolled back changes for cygpath --windows
in MSYS2 MINGW64 i got following error:
```bash
*****@rafal-laptok MINGW64 /c/rafal/git/maniekes-kafka
$ ./bin/kafka-server-start.sh config/server.properties
[0.003s][error][logging] Error opening log file 'C;C:\msys64\rafal\git\maniekes-kafka\logs\kafkaServer-gc.log;time,tags;filecount=10,filesize=100M': Invalid argument
[0.003s][error][logging] Initialization of output 'file=C;C:\msys64\rafal\git\maniekes-kafka\logs\kafkaServer-gc.log;time,tags;filecount=10,filesize=100M' using options '(null)' failed.
Invalid -Xlog option '-Xlog:gc*:file=C;C:\msys64\rafal\git\maniekes-kafka\logs\kafkaServer-gc.log;time,tags;filecount=10,filesize=100M', see error log for details.
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
```
after i add this to script it work fine(and logs are now properly displayed):
```bash
export MSYS2_ARG_CONV_EXCL="*"
# this will do the same, but only for 2 faulty arguments:
export MSYS2_ARG_CONV_EXCL="-Xlog:gc*:file=;-Dlog4j.configuration=" (since i preffer lean solutions i added this verion to script)
```
in raw MSYS2 i got following error:
```bash
*****@@rafal-laptok MSYS /c/rafal/git/maniekes-kafka
$ ./bin/kafka-server-start.sh config/server.properties
/c/rafal/git/maniekes-kafka/bin/kafka-run-class.sh: linia 352: C:\Program Files\java\jdk-19.0.1/bin/java: Bad address
/c/rafal/git/maniekes-kafka/bin/kafka-run-class.sh: linia 352: C:\Program Files\java\jdk-19.0.1/bin/java: No error
```
it was caused by this line:
https://github.com/apache/kafka/blob/trunk/bin/kafka-run-class.sh#L243
for some strange reason MSYS2 is not allowing to set `KAFKA_LOG4J_OPTS` in this way. was getting "Bad address" even if i removed usages of `KAFKA_LOG4J_OPTS` from the very bottom of the script. i tried to workaround this and finally i decided just to change name:
```bash
KAFKA_LOG4J_CMD_OPTS="-Dkafka.logs.dir=$LOG_DIR $KAFKA_LOG4J_OPTS"
```
wchich worked perfectly for all 4 shelll.
here are test results after my commit:
## git bash
```bash
*****@rafal-laptok MINGW64 /c/rafal/git/maniekes-kafka (trunk)
$ uname -a
MINGW64_NT-10.0-22621 rafal-laptok 3.3.6-341.x86_64 2022-09-05 20:28 UTC x86_64 Msys
*****@rafal-laptok MINGW64 /c/rafal/git/maniekes-kafka (trunk)
$ ./bin/kafka-server-start.sh config/server.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/tools/build/dependant-libs-2.13.10/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/trogdor/build/dependant-libs-2.13.10/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/connect/runtime/build/dependant-libs/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/connect/mirror/build/dependant-libs/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Reload4jLoggerFactory]
[2023-06-16 17:43:05,794] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)
[2023-06-16 17:43:05,950] INFO Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation (org.apache.zookeeper.common.X509Util)
[2023-06-16 17:43:06,014] INFO starting (kafka.server.KafkaServer)
[2023-06-16 17:43:06,015] INFO Connecting to zookeeper on localhost:2181 (kafka.server.KafkaServer)
[2023-06-16 17:43:06,023] INFO [ZooKeeperClient Kafka server] Initializing a new session to localhost:2181. (kafka.zookeeper.ZooKeeperClient)
[2023-06-16 17:43:06,042] INFO Client environment:zookeeper.version=3.6.4--d65253dcf68e9097c6e95a126463fd5fdeb4521c, built on 12/18/2022 18:10 GMT (org.apache.zookeeper.ZooKeeper)
[2023-06-16 17:43:06,042] INFO Client environment:host.name=192.168.1.137 (org.apache.zookeeper.ZooKeeper)
[2023-06-16 17:43:06,044] INFO Client environment:java.version=19.0.1 (org.apache.zookeeper.ZooKeeper)
[2023-06-16 17:43:06,044] INFO Client environment:java.vendor=Oracle Corporation (org.apache.zookeeper.ZooKeeper)
[2023-06-16 17:43:06,044] INFO Client environment:java.home=C:\Program Files\java\jdk-19.0.1 (org.apache.zookeeper.ZooKeeper)
(... rest of logs skipped, all works fine)
```
## MSYS2 MINGW
```bash
*****@rafal-laptok MINGW64 /c/rafal/git/maniekes-kafka
$ uname -a
MINGW64_NT-10.0-22621 rafal-laptok 3.4.6.x86_64 2023-04-01 11:43 UTC x86_64 Msys
*****@rafal-laptok MINGW64 /c/rafal/git/maniekes-kafka
$ ./bin/kafka-server-start.sh config/server.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/tools/build/dependant-libs-2.13.10/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/trogdor/build/dependant-libs-2.13.10/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/connect/runtime/build/dependant-libs/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/connect/mirror/build/dependant-libs/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Reload4jLoggerFactory]
[2023-06-16 17:44:15,893] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)
[2023-06-16 17:44:16,039] INFO Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation (org.apache.zookeeper.common.X509Util)
[2023-06-16 17:44:16,101] INFO starting (kafka.server.KafkaServer)
[2023-06-16 17:44:16,101] INFO Connecting to zookeeper on localhost:2181 (kafka.server.KafkaServer)
[2023-06-16 17:44:16,109] INFO [ZooKeeperClient Kafka server] Initializing a new session to localhost:2181. (kafka.zookeeper.ZooKeeperClient)
[2023-06-16 17:44:16,129] INFO Client environment:zookeeper.version=3.6.4--d65253dcf68e9097c6e95a126463fd5fdeb4521c, built on 12/18/2022 18:10 GMT (org.apache.zookeeper.ZooKeeper)
[2023-06-16 17:44:16,129] INFO Client environment:host.name=192.168.1.137 (org.apache.zookeeper.ZooKeeper)
[2023-06-16 17:44:16,129] INFO Client environment:java.version=19.0.1 (org.apache.zookeeper.ZooKeeper)
[2023-06-16 17:44:16,129] INFO Client environment:java.vendor=Oracle Corporation (org.apache.zookeeper.ZooKeeper)
[2023-06-16 17:44:16,129] INFO Client environment:java.home=C:\Program Files\java\jdk-19.0.1 (org.apache.zookeeper.ZooKeeper)
(... rest of logs skipped, all works fine)
```
## MSYS2
```bash
*****@rafal-laptok MSYS /c/rafal/git/maniekes-kafka
$ uname -a
MSYS_NT-10.0-22621 rafal-laptok 3.4.6.x86_64 2023-04-01 11:43 UTC x86_64 Msys
*****@rafal-laptok MSYS /c/rafal/git/maniekes-kafka
$ ./bin/kafka-server-start.sh config/server.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/tools/build/dependant-libs-2.13.10/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/trogdor/build/dependant-libs-2.13.10/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/connect/runtime/build/dependant-libs/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/connect/mirror/build/dependant-libs/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Reload4jLoggerFactory]
[2023-06-16 17:45:11,113] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)
[2023-06-16 17:45:11,269] INFO Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation (org.apache.zookeeper.common.X509Util)
[2023-06-16 17:45:11,331] INFO starting (kafka.server.KafkaServer)
[2023-06-16 17:45:11,332] INFO Connecting to zookeeper on localhost:2181 (kafka.server.KafkaServer)
[2023-06-16 17:45:11,341] INFO [ZooKeeperClient Kafka server] Initializing a new session to localhost:2181. (kafka.zookeeper.ZooKeeperClient)
[2023-06-16 17:45:11,365] INFO Client environment:zookeeper.version=3.6.4--d65253dcf68e9097c6e95a126463fd5fdeb4521c, built on 12/18/2022 18:10 GMT (org.apache.zookeeper.ZooKeeper)
[2023-06-16 17:45:11,365] INFO Client environment:host.name=192.168.1.137 (org.apache.zookeeper.ZooKeeper)
[2023-06-16 17:45:11,365] INFO Client environment:java.version=19.0.1 (org.apache.zookeeper.ZooKeeper)
[2023-06-16 17:45:11,366] INFO Client environment:java.vendor=Oracle Corporation (org.apache.zookeeper.ZooKeeper)
[2023-06-16 17:45:11,366] INFO Client environment:java.home=C:\Program Files\java\jdk-19.0.1 (org.apache.zookeeper.ZooKeeper)
(... rest of logs skipped, all works fine)
```
## CYGWIN
```bash
*****@rafal-laptok /cygdrive/c/rafal/git/maniekes-kafka
$ uname -a
CYGWIN_NT-10.0-22621 rafal-laptok 3.4.6-1.x86_64 2023-02-14 13:23 UTC x86_64 Cygwin
*****@rafal-laptok /cygdrive/c/rafal/git/maniekes-kafka
$ ./bin/kafka-server-start.sh config/server.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/tools/build/dependant-libs-2.13.10/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/trogdor/build/dependant-libs-2.13.10/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/connect/runtime/build/dependant-libs/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/connect/mirror/build/dependant-libs/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Reload4jLoggerFactory]
[2023-06-16 17:44:44,433] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)
[2023-06-16 17:44:44,580] INFO Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation (org.apache.zookeeper.common.X509Util)
[2023-06-16 17:44:44,642] INFO starting (kafka.server.KafkaServer)
[2023-06-16 17:44:44,642] INFO Connecting to zookeeper on localhost:2181 (kafka.server.KafkaServer)
[2023-06-16 17:44:44,650] INFO [ZooKeeperClient Kafka server] Initializing a new session to localhost:2181. (kafka.zookeeper.ZooKeeperClient)
[2023-06-16 17:44:44,671] INFO Client environment:zookeeper.version=3.6.4--d65253dcf68e9097c6e95a126463fd5fdeb4521c, built on 12/18/2022 18:10 GMT (org.apache.zookeeper.ZooKeeper)
[2023-06-16 17:44:44,671] INFO Client environment:host.name=192.168.1.137 (org.apache.zookeeper.ZooKeeper)
[2023-06-16 17:44:44,671] INFO Client environment:java.version=19.0.1 (org.apache.zookeeper.ZooKeeper)
[2023-06-16 17:44:44,672] INFO Client environment:java.vendor=Oracle Corporation (org.apache.zookeeper.ZooKeeper)
[2023-06-16 17:44:44,672] INFO Client environment:java.home=C:\Program Files\java\jdk-19.0.1 (org.apache.zookeeper.ZooKeeper)
(... rest of logs skipped, all works fine)
```
let me know if you want me to clarify more
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] maniekes closed pull request #13321: MINOR: added compatibility for MinGW
Posted by "maniekes (via GitHub)" <gi...@apache.org>.
maniekes closed pull request #13321: MINOR: added compatibility for MinGW
URL: https://github.com/apache/kafka/pull/13321
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] maniekes commented on pull request #13321: MINOR: added compatibility for MinGW
Posted by "maniekes (via GitHub)" <gi...@apache.org>.
maniekes commented on PR #13321:
URL: https://github.com/apache/kafka/pull/13321#issuecomment-1594651760
okay, i think i found root cause for it:
https://github.com/apache/kafka/blob/trunk/bin/kafka-server-start.sh#L25
https://github.com/apache/kafka/blob/trunk/bin/zookeeper-server-start.sh#L25
cygpath is not changing it in `kafka-run-class.sh` because it's already set in main script. the same logic is also in a few other files.
do you think we should modify it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: added compatibility for MinGW [kafka]
Posted by "maniekes (via GitHub)" <gi...@apache.org>.
maniekes commented on code in PR #13321:
URL: https://github.com/apache/kafka/pull/13321#discussion_r1372826306
##########
bin/kafka-run-class.sh:
##########
@@ -229,7 +230,7 @@ if [ -z "$KAFKA_LOG4J_OPTS" ]; then
# Log to console. This is a tool.
LOG4J_DIR="$base_dir/config/tools-log4j.properties"
# If Cygwin is detected, LOG4J_DIR is converted to Windows format.
- (( CYGWIN )) && LOG4J_DIR=$(cygpath --path --mixed "${LOG4J_DIR}")
+ (( WINDOWS_OS_FORMAT )) && LOG4J_DIR=$(cygpath --path --mixed "${LOG4J_DIR}")
KAFKA_LOG4J_OPTS="-Dlog4j.configuration=file:${LOG4J_DIR}"
Review Comment:
it's used in line 244 for setting another variable: `KAFKA_LOG4J_CMD_OPTS="-Dkafka.logs.dir=$LOG_DIR $KAFKA_LOG4J_OPTS"` which is used in line 345/347 :
` nohup "$JAVA" $KAFKA_HEAP_OPTS $KAFKA_JVM_PERFORMANCE_OPTS $KAFKA_GC_LOG_OPTS $KAFKA_JMX_OPTS $KAFKA_LOG4J_CMD_OPTS -cp "$CLASSPATH" $KAFKA_OPTS "$@" > "$CONSOLE_OUTPUT_FILE" 2>&1 < /dev/null &`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: added compatibility for MinGW [kafka]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13321:
URL: https://github.com/apache/kafka/pull/13321#issuecomment-1773983472
This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has merge conflicts, please update it with the latest from trunk (or appropriate release branch) <p> If this PR is no longer valid or desired, please feel free to close it. If no activity occurs in the next 30 days, it will be automatically closed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] divijvaidya commented on pull request #13321: MINOR: added compatibility for MinGW
Posted by "divijvaidya (via GitHub)" <gi...@apache.org>.
divijvaidya commented on PR #13321:
URL: https://github.com/apache/kafka/pull/13321#issuecomment-1592850958
Hi @maniekes
Seems like, we do indeed want to support MinGW (on a best effort basis since we can't test it) as per https://github.com/apache/kafka/blob/505c7b64875ba05301d933a9b73964fad30e8639/gradlew#L110
I will be happy to support you in merging this PR.
We need to do the following:
1. Merge from latest trunk
2. Rename the variable `CYGWIN` to a generic one since now it's enabled for `MINGW` too. Perhaps, something like `WINDOWS_OS_FORMAT`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] maniekes commented on pull request #13321: MINOR: added compatibility for MinGW
Posted by "maniekes (via GitHub)" <gi...@apache.org>.
maniekes commented on PR #13321:
URL: https://github.com/apache/kafka/pull/13321#issuecomment-1593129660
hi @divijvaidya,
i pushed changes with new variable name. since in gradlew there is support for MSYS, i have also added it for check. hope this is fine?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] maniekes commented on pull request #13321: MINOR: added compatibility for MinGW
Posted by "maniekes (via GitHub)" <gi...@apache.org>.
maniekes commented on PR #13321:
URL: https://github.com/apache/kafka/pull/13321#issuecomment-1594625896
ok, good catch with MSYS2 @divijvaidya. i have installed git bash, msys2, cygwin and tested on all of them(on msys2 i testes on msys2-mingw and on pure msys2).
what i noticed that msys2 was not reading paths properly because path translator for cygwin was adding backslashes instead of slashes (option --mixed vs --windows in cygpath, see my last commit). with option --windows it works on all those environments (hope you test is also in pipeline for cygwin to be double sure)
## Git bash
```bash
vakom@rafal-laptok MINGW64 /c/rafal/git/maniekes-kafka (trunk)
$ uname -a
MINGW64_NT-10.0-22621 rafal-laptok 3.3.6-341.x86_64 2022-09-05 20:28 UTC x86_64 Msys
vakom@rafal-laptok MINGW64 /c/rafal/git/maniekes-kafka (trunk)
$ ./bin/kafka-server-start.sh config/server.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/tools/build/dependant-libs-2.13.10/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/trogdor/build/dependant-libs-2.13.10/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/connect/runtime/build/dependant-libs/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/connect/mirror/build/dependant-libs/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Reload4jLoggerFactory]
[2023-06-16 14:35:28,564] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)
[2023-06-16 14:35:28,713] INFO Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation (org.apache.zookeeper.common.X509Util)
[2023-06-16 14:35:28,774] INFO starting (kafka.server.KafkaServer)
[2023-06-16 14:35:28,775] INFO Connecting to zookeeper on localhost:2181 (kafka.server.KafkaServer)
[2023-06-16 14:35:28,783] INFO [ZooKeeperClient Kafka server] Initializing a new session to localhost:2181. (kafka.zookeeper.ZooKeeperClient)
[2023-06-16 14:35:28,806] INFO Client environment:zookeeper.version=3.6.4--d65253dcf68e9097c6e95a126463fd5fdeb4521c, built on 12/18/2022 18:10 GMT (org.apache.zookeeper.ZooKeeper)
[2023-06-16 14:35:28,806] INFO Client environment:host.name=192.168.1.137 (org.apache.zookeeper.ZooKeeper)
[2023-06-16 14:35:28,806] INFO Client environment:java.version=19.0.1 (org.apache.zookeeper.ZooKeeper)
... (skipped rest of logs because all looks fine)
```
## MSYS2
```bash
vakom@rafal-laptok MSYS /c/rafal/git/maniekes-kafka
$ uname -a
MSYS_NT-10.0-22621 rafal-laptok 3.4.6.x86_64 2023-04-01 11:43 UTC x86_64 Msys
vakom@rafal-laptok MSYS /c/rafal/git/maniekes-kafka
$ ./bin/kafka-server-start.sh config/server.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/tools/build/dependant-libs-2.13.10/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/trogdor/build/dependant-libs-2.13.10/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/connect/runtime/build/dependant-libs/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/connect/mirror/build/dependant-libs/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Reload4jLoggerFactory]
log4j:WARN No appenders could be found for logger (kafka.utils.Log4jControllerRegistration$).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
(this is all)
```
**one remark**: in MSYS2 SLF4J is not printing logs, only warning that log appender cannot be found. however directories for kafka are being created properly and cluster works fine:
```bash
vakom@rafal-laptok MSYS /c/rafal/git/maniekes-kafka
$ ls -lart /c/tmp/kafka-logs/
razem 7
drwxr-xr-x 1 vakom vakom 0 06-16 14:41 ..
-rw-r--r-- 1 vakom vakom 0 06-16 14:41 .lock
-rw-r--r-- 1 vakom vakom 0 06-16 14:41 cleaner-offset-checkpoint
-rw-r--r-- 1 vakom vakom 0 06-16 14:41 replication-offset-checkpoint
-rw-r--r-- 1 vakom vakom 94 06-16 14:41 meta.properties
-rw-r--r-- 1 vakom vakom 6 06-16 14:42 recovery-point-offset-checkpoint
-rw-r--r-- 1 vakom vakom 6 06-16 14:42 log-start-offset-checkpoint
-rw-r--r-- 1 vakom vakom 0 06-16 14:42 .kafka_cleanshutdown
drwxr-xr-x 1 vakom vakom 0 06-16 14:42 .
```
## MSYS2 MINGW64
```bash
vakom@rafal-laptok MINGW64 /c/rafal/git/maniekes-kafka
$ uname -a
MINGW64_NT-10.0-22621 rafal-laptok 3.4.6.x86_64 2023-04-01 11:43 UTC x86_64 Msys
vakom@rafal-laptok MINGW64 /c/rafal/git/maniekes-kafka
$ ./bin/kafka-server-start.sh config/server.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/tools/build/dependant-libs-2.13.10/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/trogdor/build/dependant-libs-2.13.10/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/connect/runtime/build/dependant-libs/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/connect/mirror/build/dependant-libs/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Reload4jLoggerFactory]
log4j:WARN No appenders could be found for logger (kafka.utils.Log4jControllerRegistration$).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
(this is all)
```
**remark**: same as for pure MSYS2
## CYGWIN
```bash
vakom@rafal-laptok /cygdrive/c/rafal/git/maniekes-kafka
$ uname -a
CYGWIN_NT-10.0-22621 rafal-laptok 3.4.6-1.x86_64 2023-02-14 13:23 UTC x86_64 Cygwin
vakom@rafal-laptok /cygdrive/c/rafal/git/maniekes-kafka
$ ./bin/kafka-server-start.sh config/server.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/tools/build/dependant-libs-2.13.10/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/trogdor/build/dependant-libs-2.13.10/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/connect/runtime/build/dependant-libs/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/rafal/git/maniekes-kafka/connect/mirror/build/dependant-libs/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Reload4jLoggerFactory]
[2023-06-16 14:45:56,260] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)
[2023-06-16 14:45:56,415] INFO Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation (org.apache.zookeeper.common.X509Util)
[2023-06-16 14:45:56,484] INFO starting (kafka.server.KafkaServer)
[2023-06-16 14:45:56,485] INFO Connecting to zookeeper on localhost:2181 (kafka.server.KafkaServer)
[2023-06-16 14:45:56,497] INFO [ZooKeeperClient Kafka server] Initializing a new session to localhost:2181. (kafka.zookeeper.ZooKeeperClient)
[2023-06-16 14:45:56,520] INFO Client environment:zookeeper.version=3.6.4--d65253dcf68e9097c6e95a126463fd5fdeb4521c, built on 12/18/2022 18:10 GMT (org.apache.zookeeper.ZooKeeper)
[2023-06-16 14:45:56,520] INFO Client environment:host.name=192.168.1.137 (org.apache.zookeeper.ZooKeeper)
[2023-06-16 14:45:56,520] INFO Client environment:java.version=19.0.1 (org.apache.zookeeper.ZooKeeper)
[2023-06-16 14:45:56,520] INFO Client environment:java.vendor=Oracle Corporation (org.apache.zookeeper.ZooKeeper)
[2023-06-16 14:45:56,520] INFO Client environment:java.home=C:\Program Files\java\jdk-19.0.1 (org.apache.zookeeper.ZooKeeper)
... (skipped rest of logs because all looks fine)
```
i hope this is ok for your sanity tests.
regarding MSYS2, i think since cluster works fine and we are supporting it only occasionally it is ok to leave it like this? (without those changes it wont work at all in MSYS2 anyway...)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15685: added compatibility for MinGW [kafka]
Posted by "Zeeeeta (via GitHub)" <gi...@apache.org>.
Zeeeeta commented on PR #13321:
URL: https://github.com/apache/kafka/pull/13321#issuecomment-2058148690
My OS is RHEL 8.7 and 8.8
In 3.5.1, I ran this script with kafka.tools.GetOffsetShell just fine.
In 3.7.0, I got the following error:
Error: Could not find or load main class kafka.tools.GetOffsetShell
Caused by: java.lang.ClassNotFoundException: kafka.tools.GetOffsetShell
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: added compatibility for MinGW [kafka]
Posted by "divijvaidya (via GitHub)" <gi...@apache.org>.
divijvaidya commented on code in PR #13321:
URL: https://github.com/apache/kafka/pull/13321#discussion_r1371485060
##########
bin/kafka-run-class.sh:
##########
@@ -229,7 +230,7 @@ if [ -z "$KAFKA_LOG4J_OPTS" ]; then
# Log to console. This is a tool.
LOG4J_DIR="$base_dir/config/tools-log4j.properties"
# If Cygwin is detected, LOG4J_DIR is converted to Windows format.
- (( CYGWIN )) && LOG4J_DIR=$(cygpath --path --mixed "${LOG4J_DIR}")
+ (( WINDOWS_OS_FORMAT )) && LOG4J_DIR=$(cygpath --path --mixed "${LOG4J_DIR}")
KAFKA_LOG4J_OPTS="-Dlog4j.configuration=file:${LOG4J_DIR}"
Review Comment:
Unless I am missing something (please tell me), after setting `KAFKA_LOG4J_OPTS`, prior to this change we are using it line 345 and 347 when we start Kafka `nohup "$JAVA" $KAFKA_HEAP_OPTS $KAFKA_JVM_PERFORMANCE_OPTS $KAFKA_GC_LOG_OPTS $KAFKA_JMX_OPTS $KAFKA_LOG4J_OPTS -cp "$CLASSPATH" $KAFKA_OPTS "$@" > "$CONSOLE_OUTPUT_FILE" 2>&1 < /dev/null &`
But now, we are not using this variable over there. In that case setting it is a no-op. No?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15685: added compatibility for MinGW [kafka]
Posted by "divijvaidya (via GitHub)" <gi...@apache.org>.
divijvaidya commented on PR #13321:
URL: https://github.com/apache/kafka/pull/13321#issuecomment-1780757972
Hey @maniekes
I added a JIRA for this PR https://issues.apache.org/jira/browse/KAFKA-15685 since it is relevant to call this one out in release notes. Please feel free to assign it to yourself. I couldn't find a JIRA Id for your Github user and hence, wasn't able to do it myself.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org