You are viewing a plain text version of this content. The canonical link for it is here.

Posted to jira@kafka.apache.org by "FrankYang0529 (via GitHub)" <gi...@apache.org> on 2025/02/12 12:21:47 UTC

[PR] KAFKA-18773: Migrate the log4j1 config to log4j 2 for native image and README (wip) [kafka]

FrankYang0529 opened a new pull request, #18872:
URL: https://github.com/apache/kafka/pull/18872

   
   
   ### Committer Checklist (excluded from commit message)
   - [ ] Verify design and implementation 
   - [ ] Verify test coverage and CI build status
   - [ ] Verify documentation (including upgrade notes)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] KAFKA-18773: Migrate the log4j1 config to log4j 2 for native image and README (wip) [kafka]

Posted by "FrankYang0529 (via GitHub)" <gi...@apache.org>.

FrankYang0529 commented on PR #18872:
URL: https://github.com/apache/kafka/pull/18872#issuecomment-2659644151

Hi @dajac, I may need more time on this. Currently, there is some error message like this. I'm still check where the log from. Thanks.

```
2025-02-13T03:30:07.514257Z main ERROR Unrecognized format specifier [d]
2025-02-13T03:30:07.514271Z main ERROR Unrecognized conversion specifier [d] starting at position 16 in conversion pattern.
2025-02-13T03:30:07.514293Z main ERROR Unrecognized format specifier [thread]
2025-02-13T03:30:07.514302Z main ERROR Unrecognized conversion specifier [thread] starting at position 25 in conversion pattern.
2025-02-13T03:30:07.514318Z main ERROR Unrecognized format specifier [level]
2025-02-13T03:30:07.514331Z main ERROR Unrecognized conversion specifier [level] starting at position 35 in conversion pattern.
2025-02-13T03:30:07.514347Z main ERROR Unrecognized format specifier [logger]
2025-02-13T03:30:07.514361Z main ERROR Unrecognized conversion specifier [logger] starting at position 47 in conversion pattern.
2025-02-13T03:30:07.514376Z main ERROR Unrecognized format specifier [msg]
2025-02-13T03:30:07.514391Z main ERROR Unrecognized conversion specifier [msg] starting at position 54 in conversion pattern.
2025-02-13T03:30:07.514406Z main ERROR Unrecognized format specifier [n]
2025-02-13T03:30:07.514420Z main ERROR Unrecognized conversion specifier [n] starting at position 56 in conversion pattern.
```

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] KAFKA-18773: Migrate the log4j1 config to log4j 2 for native image and README [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.

chia7712 merged PR #18872:
URL: https://github.com/apache/kafka/pull/18872


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] KAFKA-18773: Migrate the log4j1 config to log4j 2 for native image and README (wip) [kafka]

Posted by "dajac (via GitHub)" <gi...@apache.org>.

dajac commented on PR #18872:
URL: https://github.com/apache/kafka/pull/18872#issuecomment-2662677705

   @FrankYang0529 Thanks. Do you think that we could merge it this week?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] KAFKA-18773: Migrate the log4j1 config to log4j 2 for native image and README [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.

chia7712 commented on PR #18872:
URL: https://github.com/apache/kafka/pull/18872#issuecomment-2666287853

   cherry-pick to 4.0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] KAFKA-18773: Migrate the log4j1 config to log4j 2 for native image and README (wip) [kafka]

Posted by "dajac (via GitHub)" <gi...@apache.org>.

dajac commented on PR #18872:
URL: https://github.com/apache/kafka/pull/18872#issuecomment-2657562181

   @FrankYang0529 How is it going with this one?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] KAFKA-18773: Migrate the log4j1 config to log4j 2 for native image and README (wip) [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.

chia7712 commented on PR #18872:
URL: https://github.com/apache/kafka/pull/18872#issuecomment-2663825547

   the failure may be related to https://github.com/apache/logging-log4j2/issues/1539 - I will test the 2.25.0-SNAPSHOT locally.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] KAFKA-18773: Migrate the log4j1 config to log4j 2 for native image and README (wip) [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.

chia7712 commented on code in PR #18872:
URL: https://github.com/apache/kafka/pull/18872#discussion_r1959224906


##########
docker/native/launch:
##########
@@ -42,11 +42,11 @@ result=$(/opt/kafka/kafka.Kafka setup \
       --default-configs-dir /etc/kafka/docker \
       --mounted-configs-dir /mnt/shared/config \
       --final-configs-dir /opt/kafka/config \
-      -Dlog4j.configuration=file:/opt/kafka/config/tools-log4j.properties 2>&1) || \
+      -Dlog4j2.configurationFile=file:/opt/kafka/config/tools-log4j2.yaml 2>&1) || \

Review Comment:
   `/opt/kafka/config/tools-log4j2.yaml` is nonexistent in running the setup. Maybe we can just remove it to align the command to jvm dockerfile



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] KAFKA-18773: Migrate the log4j1 config to log4j 2 for native image and README [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.

chia7712 commented on PR #18872:
URL: https://github.com/apache/kafka/pull/18872#issuecomment-2666267732

   the flaky is traced by https://issues.apache.org/jira/browse/KAFKA-15900


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] KAFKA-18773: Migrate the log4j1 config to log4j 2 for native image and README [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.

chia7712 commented on code in PR #18872:
URL: https://github.com/apache/kafka/pull/18872#discussion_r1960176376


##########
build.gradle:
##########
@@ -1072,7 +1072,6 @@ project(':core') {
     implementation libs.argparse4j
     implementation libs.commonsValidator
     implementation libs.jacksonDatabind
-    implementation libs.jacksonModuleScala

Review Comment:
   #18950



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] KAFKA-18773: Migrate the log4j1 config to log4j 2 for native image and README (wip) [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.

chia7712 commented on PR #18872:
URL: https://github.com/apache/kafka/pull/18872#issuecomment-2664812516

ok, I run the native image successfully on my local. There are many issues we need to fix.

@FrankYang0529 Could you please try following steps on your PR.

1. use following `reflect-config.json` and `resource-config.json`. They are generated by `Tracing Agent` (see https://www.graalvm.org/22.2/reference-manual/native-image/metadata/AutomaticMetadataCollection/#tracing-agent)

[resource-config.json](https://github.com/user-attachments/files/18840596/resource-config.json)
[reflect-config.json](https://github.com/user-attachments/files/18840597/reflect-config.json)

2. remove `jacksonModuleScala` from dependency. it has issue on https://github.com/FasterXML/jackson-module-scala/issues/621 but we don't use it actually.

3. fix the bug according to my comment (https://github.com/apache/kafka/pull/18394/files#r1959192770)

Additionally, the storage command has issue and it is handled by #18844 - hence, this PR can't fix native image totally until #18844 gets fixed - however, this PR can adopt the solution temporarily.

> Do you think that we could merge it this week?

@dajac yes, I think it can get fixed this week

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] KAFKA-18773: Migrate the log4j1 config to log4j 2 for native image and README [kafka]

Posted by "FrankYang0529 (via GitHub)" <gi...@apache.org>.

FrankYang0529 commented on PR #18872:
URL: https://github.com/apache/kafka/pull/18872#issuecomment-2665362153

   Hi @chia7712, thanks for the review. I also update the document about generating reachability metadata. If we need new files next time, we can follow it to generate metadata.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] KAFKA-18773: Migrate the log4j1 config to log4j 2 for native image and README [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.

chia7712 commented on code in PR #18872:
URL: https://github.com/apache/kafka/pull/18872#discussion_r1960125365


##########
build.gradle:
##########
@@ -1072,7 +1072,6 @@ project(':core') {
     implementation libs.argparse4j
     implementation libs.commonsValidator
     implementation libs.jacksonDatabind
-    implementation libs.jacksonModuleScala

Review Comment:
   for another, we will file a PR to fix the license check. The PR #18004 also introduces a license warning.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] KAFKA-18773: Migrate the log4j1 config to log4j 2 for native image and README [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.

chia7712 commented on code in PR #18872:
URL: https://github.com/apache/kafka/pull/18872#discussion_r1959927541


##########
build.gradle:
##########
@@ -1072,7 +1072,6 @@ project(':core') {
     implementation libs.argparse4j
     implementation libs.commonsValidator
     implementation libs.jacksonDatabind
-    implementation libs.jacksonModuleScala

Review Comment:
   for future reader, the dependency was introduced by https://github.com/apache/kafka/commit/47a9871ef65a13f9d58d5ea216de340f7e123da5, and https://github.com/apache/kafka/commit/b6183a41342c765daec9c88f9d2723a221131960 rewrite the tool by java so we don't need to use `jackson-module-scala` anymore.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org