You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/06/12 13:29:29 UTC
[GitHub] [pulsar] nlu90 opened a new pull request, #15728: [Function] provide default error handler for function log appender
nlu90 opened a new pull request, #15728:
URL: https://github.com/apache/pulsar/pull/15728
<!--
### Contribution Checklist
- PR title format should be *[type][component] summary*. For details, see *[Guideline - Pulsar PR Naming Convention](https://docs.google.com/document/d/1d8Pw6ZbWk-_pCKdOmdvx9rnhPiyuxwq60_TrD68d7BA/edit#heading=h.trs9rsex3xom)*.
- Fill out the template below to describe the changes contributed by the pull request. That will give reviewers the context they need to do the review.
- Each pull request should address only one issue, not mix up code from multiple issues.
- Each commit in the pull request has a meaningful commit message
- Once all items of the checklist are addressed, remove the above text and this checklist, leaving only the filled out template below.
**(The sections below can be removed for hotfixes of typos)**
-->
### Motivation
Fixes #15274
### Modifications
1. create a DefaultErrorHandler during LogAppender initialization.
### Verifying this change
- [ ] Make sure that the change passes the CI checks.
*(Please pick either of the following options)*
This change is a trivial rework / code cleanup without any test coverage.
*(or)*
This change is already covered by existing tests, such as *(please describe tests)*.
*(or)*
This change added tests and can be verified as follows:
*(example:)*
- *Added integration tests for end-to-end deployment with large payloads (10MB)*
- *Extended integration test for recovery after broker failure*
### Does this pull request potentially affect one of the following parts:
*If `yes` was chosen, please highlight the changes*
- Dependencies (does it add or upgrade a dependency): (yes / no)
- The public API: (yes / no)
- The schema: (yes / no / don't know)
- The default values of configurations: (yes / no)
- The wire protocol: (yes / no)
- The rest endpoints: (yes / no)
- The admin cli options: (yes / no)
- Anything that affects deployment: (yes / no / don't know)
### Documentation
Check the box below or label this PR directly.
Need to update docs?
- [ ] `doc-required`
(Your PR needs to update docs and you will update later)
- [X] `no-need-doc`
Internal bug fix
- [ ] `doc`
(Your PR contains doc changes)
- [ ] `doc-added`
(Docs have been already added)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui closed pull request #15728: [Function] provide default error handler for function log appender
Posted by GitBox <gi...@apache.org>.
codelipenghui closed pull request #15728: [Function] provide default error handler for function log appender
URL: https://github.com/apache/pulsar/pull/15728
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] nlu90 commented on a diff in pull request #15728: [Function] provide default error handler
Posted by GitBox <gi...@apache.org>.
nlu90 commented on code in PR #15728:
URL: https://github.com/apache/pulsar/pull/15728#discussion_r879953842
##########
pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/LogAppender.java:
##########
@@ -47,15 +54,16 @@ public LogAppender(PulsarClient pulsarClient, String logTopic, String fqn, Strin
this.logTopic = logTopic;
this.fqn = fqn;
this.instance = instance;
+ this.errorHandler = new DefaultErrorHandler(this);
Review Comment:
I first tried to implement our own `LogErrorHandler` which also sends error messages and throwables to the pulsar cluster using appender's producer.
But since the producer also logs debug or error message during execution, I met an recursive call issue as follows:
```
2022-05-23 15:45:55,433 public/default/LoggingFunction-0 WARN org.apache.logging.slf4j.Log4jLogger caught java.lang.StackOverflowError logging ParameterizedMessage: [{}] [{}] add message to batch, num messages in batch so far {} java.lang.StackOverflowError
at org.apache.pulsar.client.impl.ProducerImpl.sendAsync(ProducerImpl.java:565)
at org.apache.pulsar.client.impl.ProducerImpl.internalSendAsync(ProducerImpl.java:340)
at org.apache.pulsar.client.impl.TypedMessageBuilderImpl.sendAsync(TypedMessageBuilderImpl.java:101)
at org.apache.pulsar.functions.instance.LogAppender$LogErrorHandler.error(LogAppender.java:156)
at org.apache.logging.log4j.core.config.AppenderControl.appenderErrorHandlerMessage(AppenderControl.java:118)
at org.apache.logging.log4j.core.config.AppenderControl.isRecursiveCall(AppenderControl.java:110)
at org.apache.logging.log4j.core.config.AppenderControl.shouldSkip(AppenderControl.java:93)
at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:86)
at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:542)
at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:500)
at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:483)
at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417)
at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:82)
at org.apache.logging.log4j.core.Logger.log(Logger.java:161)
at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2205)
at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2159)
at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2142)
at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2028)
at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1891)
at org.apache.logging.slf4j.Log4jLogger.debug(Log4jLogger.java:134)
at org.apache.pulsar.client.impl.BatchMessageContainerImpl.add(BatchMessageContainerImpl.java:76)
at org.apache.pulsar.client.impl.ProducerImpl.serializeAndSendMessage(ProducerImpl.java:641)
at org.apache.pulsar.client.impl.ProducerImpl.sendAsync(ProducerImpl.java:555)
at org.apache.pulsar.client.impl.ProducerImpl.internalSendAsync(ProducerImpl.java:340)
at org.apache.pulsar.client.impl.TypedMessageBuilderImpl.sendAsync(TypedMessageBuilderImpl.java:101)
at org.apache.pulsar.functions.instance.LogAppender$LogErrorHandler.error(LogAppender.java:156)
at org.apache.logging.log4j.core.config.AppenderControl.appenderErrorHandlerMessage(AppenderControl.java:118)
at org.apache.logging.log4j.core.config.AppenderControl.isRecursiveCall(AppenderControl.java:110)
at org.apache.logging.log4j.core.config.AppenderControl.shouldSkip(AppenderControl.java:93)
at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:86)
at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:542)
at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:500)
at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:483)
at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417)
at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:82)
at org.apache.logging.log4j.core.Logger.log(Logger.java:161)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] mattisonchao merged pull request #15728: [Function] provide default error handler for function log appender
Posted by GitBox <gi...@apache.org>.
mattisonchao merged PR #15728:
URL: https://github.com/apache/pulsar/pull/15728
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] nlu90 commented on pull request #15728: [Function] provide default error handler
Posted by GitBox <gi...@apache.org>.
nlu90 commented on PR #15728:
URL: https://github.com/apache/pulsar/pull/15728#issuecomment-1136506267
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] nlu90 commented on pull request #15728: [Function] provide default error handler
Posted by GitBox <gi...@apache.org>.
nlu90 commented on PR #15728:
URL: https://github.com/apache/pulsar/pull/15728#issuecomment-1135262715
@freeznet @liudezhi2098 Could you help take a look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on pull request #15728: [Function] provide default error handler for function log appender
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on PR #15728:
URL: https://github.com/apache/pulsar/pull/15728#issuecomment-1150677078
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org