You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2022/02/16 03:22:09 UTC
[GitHub] [kafka] vvcephei opened a new pull request #11767: KAFKA-13598: set log4j appender to default acks
vvcephei opened a new pull request #11767:
URL: https://github.com/apache/kafka/pull/11767
After https://github.com/apache/kafka/pull/11691, the
KafkaLog4JAppender causes the process to crash due
to the mismatch between the new produce default
idempotent mode and the appender's overridden
`acks=1` property.
* Removes the `acks=1` override, so the appender falls
back on the producer's default acks config.
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] vvcephei commented on pull request #11767: KAFKA-13598: set log4j appender to default acks
Posted by GitBox <gi...@apache.org>.
vvcephei commented on pull request #11767:
URL: https://github.com/apache/kafka/pull/11767#issuecomment-1041059391
This issue surfaced in Confluent's ksqlDB. Once ksqlDB started to build against Kafka post-#11691, it crashes with:
```
Exception in thread "main" java.lang.ExceptionInInitializerError
at org.slf4j.impl.Log4jLoggerFactory.<init>(Log4jLoggerFactory.java:66)
at org.slf4j.impl.StaticLoggerBinder.<init>(StaticLoggerBinder.java:72)
at org.slf4j.impl.StaticLoggerBinder.<clinit>(StaticLoggerBinder.java:45)
at org.slf4j.LoggerFactory.bind(LoggerFactory.java:150)
at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:124)
at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:417)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:362)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:388)
at io.confluent.ksql.rest.server.KsqlServerMain.<clinit>(KsqlServerMain.java:39)
Caused by: org.apache.kafka.common.config.ConfigException: Must set acks to all in order to use the idempotent producer. Otherwise we cannot guarantee idempotence.
at org.apache.kafka.clients.producer.ProducerConfig.postProcessAndValidateIdempotenceConfigs(ProducerConfig.java:475)
at org.apache.kafka.clients.producer.ProducerConfig.postProcessParsedConfig(ProducerConfig.java:443)
at org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:114)
at org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:133)
at org.apache.kafka.clients.producer.ProducerConfig.<init>(ProducerConfig.java:511)
at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:289)
at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:316)
at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:301)
at org.apache.kafka.log4jappender.KafkaLog4jAppender.getKafkaProducer(KafkaLog4jAppender.java:341)
at org.apache.kafka.log4jappender.KafkaLog4jAppender.activateOptions(KafkaLog4jAppender.java:335)
at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172)
at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104)
at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:842)
at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768)
at org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:672)
at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:516)
at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580)
at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526)
at org.apache.log4j.LogManager.<clinit>(LogManager.java:127)
```
I would expect any other project using the `KafkaLog4JAppender` to experience the same behavior.
I'm not sure why the KafkaLog4JAppender originally overrode the acks to 1, but it's certainly incompatible with the new idempotence default. An alternative would be to leave acks=1 and disable idempotence, but I figured fewer overrides make the appender easier to reason about.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] hachikuji commented on pull request #11767: KAFKA-13598: set log4j appender to default acks
Posted by GitBox <gi...@apache.org>.
hachikuji commented on pull request #11767:
URL: https://github.com/apache/kafka/pull/11767#issuecomment-1042497943
> Yes, I don't know the getters serve any purpose, either. But I'm still worried that there's someone relying on it.
I'd like to hear what @hachikuji 's thought since he did the change to make `requiredNumAcks` to 1 in this PR: https://github.com/apache/kafka/pull/5425#discussion_r205861230
Unfair to mention a PR from 4 years ago 😉 . Regardless whether anyone actually uses this class, it does seem weird to have getters that we know are broken. It does seem better to initialize the defaults explicitly. Perhaps we may as well fix the other broken getters while we're at it. At a quick glance, it looks like `maxBlockMs` is the only other one?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] vvcephei commented on pull request #11767: KAFKA-13598: set log4j appender to default acks
Posted by GitBox <gi...@apache.org>.
vvcephei commented on pull request #11767:
URL: https://github.com/apache/kafka/pull/11767#issuecomment-1041078717
Thanks, @showuon ! I noticed that also, but there are also a whole bunch of other nullable object references with primitive-typed getters in that class, so I figured it was just some funky log4j thing.
I figured out that the setter is used by log4 to apply the configuration (for example, `log4j.appender.kafka_appender.RequiredNumAcks=-1` calls `setRequredNumAcks(-1)`), but I don't know if the getters serve any purpose.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] vvcephei commented on pull request #11767: KAFKA-13598: set log4j appender to default acks
Posted by GitBox <gi...@apache.org>.
vvcephei commented on pull request #11767:
URL: https://github.com/apache/kafka/pull/11767#issuecomment-1043135161
Closing in favor of https://issues.apache.org/jira/browse/KAFKA-13673 / https://github.com/apache/kafka/pull/11788
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] vvcephei closed pull request #11767: KAFKA-13598: set log4j appender to default acks
Posted by GitBox <gi...@apache.org>.
vvcephei closed pull request #11767:
URL: https://github.com/apache/kafka/pull/11767
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] hachikuji edited a comment on pull request #11767: KAFKA-13598: set log4j appender to default acks
Posted by GitBox <gi...@apache.org>.
hachikuji edited a comment on pull request #11767:
URL: https://github.com/apache/kafka/pull/11767#issuecomment-1042497943
> Yes, I don't know the getters serve any purpose, either. But I'm still worried that there's someone relying on it.
I'd like to hear what @hachikuji 's thought since he did the change to make `requiredNumAcks` to 1 in this PR: https://github.com/apache/kafka/pull/5425#discussion_r205861230
Unfair to mention a PR from 4 years ago 😉 . Regardless whether anyone actually uses this class, it does seem weird to have getters that we know are broken. I agree it seems better to initialize the defaults explicitly (and to use types consistently). Perhaps we may as well fix the other broken getters while we're at it. At a quick glance, it looks like `maxBlockMs` is the only other one?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] showuon commented on pull request #11767: KAFKA-13598: set log4j appender to default acks
Posted by GitBox <gi...@apache.org>.
showuon commented on pull request #11767:
URL: https://github.com/apache/kafka/pull/11767#issuecomment-1041150174
Yes, I don't know the getters serve any purpose, either. But I'm still worried that there's someone relying on it.
I'd like to hear what @hachikuji 's thought since he did the change to make `requiredNumAcks` to 1 in this PR: https://github.com/apache/kafka/pull/5425#discussion_r205861230
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org