You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@kyuubi.apache.org by "imperio-wxm (via GitHub)" <gi...@apache.org> on 2023/02/09 09:38:10 UTC
[GitHub] [kyuubi] imperio-wxm opened a new issue, #4282: [Bug] Caused by: java.util.ConcurrentModificationException
imperio-wxm opened a new issue, #4282:
URL: https://github.com/apache/kyuubi/issues/4282
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
### Search before asking
- [X] I have searched in the [issues](https://github.com/apache/kyuubi/issues?q=is%3Aissue) and found no similar issues.
### Describe the bug
When using log4j2 to output logs asynchronously, an error is reported.
- sbt
```
"org.apache.logging.log4j" % "log4j-api" % "2.19.0",
"org.apache.logging.log4j" % "log4j-core" % "2.19.0",
"org.slf4j" % "slf4j-api" % "2.0.6",
"org.apache.logging.log4j" % "log4j-slf4j-impl" % "2.19.0",
"org.slf4j" % "slf4j-reload4j" % "2.0.6",
"com.lmax" % "disruptor" % "3.4.4"sbt
```
- error log
```java
"Caused by: java.util.ConcurrentModificationException",
"\tat java.util.Hashtable$Enumerator.next(Hashtable.java:1378)",
"\tat org.apache.logging.log4j.message.ParameterFormatter.appendMap(ParameterFormatter.java:581)",
"\tat org.apache.logging.log4j.message.ParameterFormatter.appendPotentiallyRecursiveValue(ParameterFormatter.java:512)",
"\tat org.apache.logging.log4j.message.ParameterFormatter.recursiveDeepToString(ParameterFormatter.java:477)",
"\tat org.apache.logging.log4j.message.ParameterFormatter.recursiveDeepToString(ParameterFormatter.java:448)",
"\tat org.apache.logging.log4j.message.ParameterFormatter.formatMessage2(ParameterFormatter.java:191)",
"\tat org.apache.logging.log4j.message.ParameterizedMessage.formatTo(ParameterizedMessage.java:227)",
"\tat org.apache.logging.log4j.core.pattern.MessagePatternConverter$SimpleMessagePatternConverter.format(MessagePatternConverter.java:121)",
"\tat org.apache.logging.log4j.core.pattern.PatternFormatter.format(PatternFormatter.java:44)",
"\tat org.apache.logging.log4j.core.layout.PatternLayout$PatternFormatterPatternSerializer.toSerializable(PatternLayout.java:384)",
"\tat org.apache.logging.log4j.core.layout.PatternLayout$PatternFormatterPatternSerializer.toSerializable(PatternLayout.java:375)",
"\tat org.apache.logging.log4j.core.layout.PatternLayout.toSerializable(PatternLayout.java:216)",
"\tat org.apache.logging.log4j.core.layout.PatternLayout.toSerializable(PatternLayout.java:59)",
"\tat org.apache.logging.log4j.core.appender.AbstractWriterAppender.append(AbstractWriterAppender.java:101)",
"\tat org.apache.kyuubi.operation.log.Log4j2DivertAppender.append(Log4j2DivertAppender.scala:77)",
"\tat org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:161)",
"\t... 20 more"
```
- log4j2.xml
```xml
<?xml version="1.0" encoding="UTF-8"?>
<!--monitorInterval：Log4j能够自动检测修改配置 文件和重新配置本身，设置间隔秒数-->
<configuration status="INFO" monitorInterval="30">
<appenders>
<console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"/>
</console>
<RollingFile name="RollingFileInfo" fileName="/opt/spark/logs/info.log"
filePattern="/opt/spark/logs/${date:yyyy-MM}/info-%d{yyyy-MM-dd}-%i.log">
<ThresholdFilter level="info" onMatch="ACCEPT" onMismatch="DENY"/>
<PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"/>
<Policies>
<TimeBasedTriggeringPolicy/>
<SizeBasedTriggeringPolicy size="100 MB"/>
</Policies>
<DefaultRolloverStrategy max="20"/>
</RollingFile>
</appenders>
<loggers>
<logger name="org.springframework" level="INFO"/>
<AsyncLogger name="RollingFileInfo" level="info" additivity="false">
<appender-ref ref="RollingFileInfo"/>
</AsyncLogger>
<AsyncLogger name="Console" level="info" additivity="false">
<appender-ref ref="Console"/>
</AsyncLogger>
<root level="INFO">
<appender-ref ref="Console"/>
<appender-ref ref="RollingFileInfo"/>
</root>
</loggers>
</configuration>
```
### Affects Version(s)
master
### Kyuubi Server Log Output
_No response_
### Kyuubi Engine Log Output
_No response_
### Kyuubi Server Configurations
_No response_
### Kyuubi Engine Configurations
_No response_
### Additional context
_No response_
### Are you willing to submit PR?
- [ ] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to fix.
- [ ] No. I cannot submit a PR at this time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] github-actions[bot] commented on issue #4282: [Bug] Caused by: java.util.ConcurrentModificationException
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #4282:
URL: https://github.com/apache/kyuubi/issues/4282#issuecomment-1423895318
Hello @imperio-wxm,
Thanks for finding the time to report the issue!
We really appreciate the community's efforts to improve Apache Kyuubi.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] imperio-wxm commented on issue #4282: [Bug] ConcurrentModificationException when log4j2 async enabled
Posted by "imperio-wxm (via GitHub)" <gi...@apache.org>.
imperio-wxm commented on issue #4282:
URL: https://github.com/apache/kyuubi/issues/4282#issuecomment-1434051887
@pan3793 The new code has been verified and works fine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] pan3793 closed issue #4282: [Bug] ConcurrentModificationException when log4j2 async enabled
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 closed issue #4282: [Bug] ConcurrentModificationException when log4j2 async enabled
URL: https://github.com/apache/kyuubi/issues/4282
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] pan3793 commented on issue #4282: [Bug] ConcurrentModificationException when log4j2 async enabled
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on issue #4282:
URL: https://github.com/apache/kyuubi/issues/4282#issuecomment-1425198103
Thanks for reporting this issue, I have sent PR#4300 to fix it, would you please help to test it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] pan3793 commented on issue #4282: [Bug] ConcurrentModificationException when log4j2 async enabled
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on issue #4282:
URL: https://github.com/apache/kyuubi/issues/4282#issuecomment-1429039038
`synchronized(writer)` is quite cheap, I decided to move to use `writeLock` instead for better semantics, would you please have another try?
https://github.com/apache/kyuubi/pull/4300#discussion_r1105218890
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] imperio-wxm commented on issue #4282: [Bug] ConcurrentModificationException when log4j2 async enabled
Posted by "imperio-wxm (via GitHub)" <gi...@apache.org>.
imperio-wxm commented on issue #4282:
URL: https://github.com/apache/kyuubi/issues/4282#issuecomment-1425837808
@pan3793 Thanks for the fix, I tested it and no more errors. But I found in the submission that you added a lock in the append method, will this affect performance?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] pan3793 commented on issue #4282: [Bug] ConcurrentModificationException when log4j2 async enabled
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on issue #4282:
URL: https://github.com/apache/kyuubi/issues/4282#issuecomment-1432824098
@imperio-wxm would you please help to verify #4300 again?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org