You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by "CodeSmell (via GitHub)" <gi...@apache.org> on 2023/11/06 17:24:24 UTC

[PR] BreakOnFirstError with extra logging [camel]

CodeSmell opened a new pull request, #11920:
URL: https://github.com/apache/camel/pull/11920

   # Description
   
   This PR has some extra logging related to CAMEL-20044 
   This PR doesn't match the log files in the issue but is similar
   This PR is meant as discussion point to work on issues in `breakOnFirstError` processing
   
   # Target
   
   Camel-3.21.x
   
   # Tracking
   CAMEL-20044 
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-20044 add extra logging for BreakOnFirstError [camel]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11920:
URL: https://github.com/apache/camel/pull/11920#issuecomment-1802371276

   :no_entry_sign: There are (likely) no changes in core core to be tested in this PR


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-20044 add extra logging for BreakOnFirstError [camel]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11920:
URL: https://github.com/apache/camel/pull/11920#issuecomment-1799335283

   :no_entry_sign: There are (likely) no changes in core core to be tested in this PR


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BreakOnFirstError with extra logging [camel]

Posted by "orpiske (via GitHub)" <gi...@apache.org>.
orpiske commented on PR #11920:
URL: https://github.com/apache/camel/pull/11920#issuecomment-1798156665

   Looking at it ...


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-20044 add extra logging for BreakOnFirstError [camel]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11920:
URL: https://github.com/apache/camel/pull/11920#issuecomment-1799303263

   :no_entry_sign: There are (likely) no changes in core core to be tested in this PR


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-20044 add extra logging for BreakOnFirstError [camel]

Posted by "orpiske (via GitHub)" <gi...@apache.org>.
orpiske commented on code in PR #11920:
URL: https://github.com/apache/camel/pull/11920#discussion_r1384928509


##########
components/camel-kafka/src/main/java/org/apache/camel/component/kafka/consumer/support/KafkaRecordProcessor.java:
##########
@@ -112,30 +115,48 @@ public ProcessingResult processExchange(
             exchange.setException(e);
         }
         if (exchange.getException() != null) {
-            boolean breakOnErrorExit = processException(exchange, partition, lastResult.getPartitionLastOffset(),
+            
+            LOG.debug("an exception was thrown for record at partition {} and offset {}",
+                    record.partition(), record.offset());
+            
+            boolean breakOnErrorExit = processException(exchange, topicPartition, record, lastResult,
                     exceptionHandler);
-            return new ProcessingResult(breakOnErrorExit, lastResult.getPartitionLastOffset(), true);
+            
+            return new ProcessingResult(breakOnErrorExit, lastResult.getPartition(), lastResult.getPartitionLastOffset(), true);
         } else {
-            return new ProcessingResult(false, record.offset(), exchange.getException() != null);
+            return new ProcessingResult(false, record.partition(), record.offset(), exchange.getException() != null);
         }
     }
 
     private boolean processException(
-            Exchange exchange, TopicPartition partition, long partitionLastOffset,
+            Exchange exchange, TopicPartition topicPartition, 
+            ConsumerRecord<Object, Object> record, ProcessingResult lastResult,
             ExceptionHandler exceptionHandler) {
 
         // processing failed due to an unhandled exception, what should we do
         if (configuration.isBreakOnFirstError()) {
+            
+            if (lastResult.getPartition() != -1 &&
+                lastResult.getPartition() != record.partition()) {
+                LOG.error("about to process an exception with UNEXPECTED partition & offset. Got topic partition {}. " + 
+                        " The last result was on partition {} with offset {} but was expecting partition {} with offset {}",
+                        topicPartition.partition(), lastResult.getPartition(), lastResult.getPartitionLastOffset(), 
+                        record.partition(), record.offset());
+            }
+            
             // we are failing and we should break out
             if (LOG.isWarnEnabled()) {
-                LOG.warn("Error during processing {} from topic: {}", exchange, partition.topic(), exchange.getException());
-                LOG.warn("Will seek consumer to offset {} and start polling again.", partitionLastOffset);
+                LOG.warn("Error during processing {} from topic: {}", exchange, topicPartition.topic(), exchange.getException());

Review Comment:
   Just one more note: the code as it is, is still not printing the exception message. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-20044 add extra logging for BreakOnFirstError [camel]

Posted by "orpiske (via GitHub)" <gi...@apache.org>.
orpiske merged PR #11920:
URL: https://github.com/apache/camel/pull/11920


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-20044 add extra logging for BreakOnFirstError [camel]

Posted by "CodeSmell (via GitHub)" <gi...@apache.org>.
CodeSmell commented on PR #11920:
URL: https://github.com/apache/camel/pull/11920#issuecomment-1802009407

   > This is looking OK.
   
   @orpiske 
   I updated the logging here so that it each word is capitalized per other comment 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-20044 add extra logging for BreakOnFirstError [camel]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11920:
URL: https://github.com/apache/camel/pull/11920#issuecomment-1802115561

   ### Components test results:
   
   | Total | Tested | Failed :x: | Passed :white_check_mark: | 
   | --- | --- | --- |  --- |
   | 2 | 2 | 1 | 1 |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-20044 add extra logging for BreakOnFirstError [camel]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11920:
URL: https://github.com/apache/camel/pull/11920#issuecomment-1802371246

   ### Components test results:
   
   | Total | Tested | Failed :x: | Passed :white_check_mark: | 
   | --- | --- | --- |  --- |
   | 2 | 2 | 0 | 2 |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BreakOnFirstError with extra logging [camel]

Posted by "CodeSmell (via GitHub)" <gi...@apache.org>.
CodeSmell commented on PR #11920:
URL: https://github.com/apache/camel/pull/11920#issuecomment-1795659932

   Wasn't sure if should checkin these files under src/generated/resources
   
   - kafka.json
   - spring-main.json
   - component.properties
   - other.properties


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-20044 add extra logging for BreakOnFirstError [camel]

Posted by "orpiske (via GitHub)" <gi...@apache.org>.
orpiske commented on PR #11920:
URL: https://github.com/apache/camel/pull/11920#issuecomment-1801458023

   This is looking OK. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-20044 add extra logging for BreakOnFirstError [camel]

Posted by "orpiske (via GitHub)" <gi...@apache.org>.
orpiske commented on PR #11920:
URL: https://github.com/apache/camel/pull/11920#issuecomment-1802016189

   Thank you @CodeSmell! I am now running manual tests on this code and will merge / report back soon.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BreakOnFirstError with extra logging [camel]

Posted by "orpiske (via GitHub)" <gi...@apache.org>.
orpiske commented on code in PR #11920:
URL: https://github.com/apache/camel/pull/11920#discussion_r1384653079


##########
components/camel-kafka/src/main/java/org/apache/camel/component/kafka/consumer/support/KafkaRecordProcessor.java:
##########
@@ -112,30 +115,48 @@ public ProcessingResult processExchange(
             exchange.setException(e);
         }
         if (exchange.getException() != null) {
-            boolean breakOnErrorExit = processException(exchange, partition, lastResult.getPartitionLastOffset(),
+            
+            LOG.debug("an exception was thrown for record at partition {} and offset {}",
+                    record.partition(), record.offset());
+            
+            boolean breakOnErrorExit = processException(exchange, topicPartition, record, lastResult,
                     exceptionHandler);
-            return new ProcessingResult(breakOnErrorExit, lastResult.getPartitionLastOffset(), true);
+            
+            return new ProcessingResult(breakOnErrorExit, lastResult.getPartition(), lastResult.getPartitionLastOffset(), true);
         } else {
-            return new ProcessingResult(false, record.offset(), exchange.getException() != null);
+            return new ProcessingResult(false, record.partition(), record.offset(), exchange.getException() != null);
         }
     }
 
     private boolean processException(
-            Exchange exchange, TopicPartition partition, long partitionLastOffset,
+            Exchange exchange, TopicPartition topicPartition, 
+            ConsumerRecord<Object, Object> record, ProcessingResult lastResult,
             ExceptionHandler exceptionHandler) {
 
         // processing failed due to an unhandled exception, what should we do
         if (configuration.isBreakOnFirstError()) {
+            
+            if (lastResult.getPartition() != -1 &&
+                lastResult.getPartition() != record.partition()) {
+                LOG.error("about to process an exception with UNEXPECTED partition & offset. Got topic partition {}. " + 
+                        " The last result was on partition {} with offset {} but was expecting partition {} with offset {}",
+                        topicPartition.partition(), lastResult.getPartition(), lastResult.getPartitionLastOffset(), 
+                        record.partition(), record.offset());
+            }
+            
             // we are failing and we should break out
             if (LOG.isWarnEnabled()) {
-                LOG.warn("Error during processing {} from topic: {}", exchange, partition.topic(), exchange.getException());
-                LOG.warn("Will seek consumer to offset {} and start polling again.", partitionLastOffset);
+                LOG.warn("Error during processing {} from topic: {}", exchange, topicPartition.topic(), exchange.getException());

Review Comment:
   Maybe also log the exception message for more details? 
   
   ```
   LOG.warn("Error during processing exchange {} from topic {}: {}", exchange, topicPartition.topic(), exchange.getException() ? null : exchange.getException().getMessage(), exchange.getException());
   ```



##########
components/camel-kafka/src/main/java/org/apache/camel/component/kafka/consumer/support/ProcessingResult.java:
##########
@@ -21,14 +21,16 @@
 
 public final class ProcessingResult {
     private static final ProcessingResult UNPROCESSED_RESULT
-            = new ProcessingResult(false, AbstractCommitManager.START_OFFSET, false);
+            = new ProcessingResult(false, -1, AbstractCommitManager.START_OFFSET, false);

Review Comment:
   Please, use a constant here and document it. That way it may be easier for us to understand the code later on.



##########
components/camel-kafka/src/main/java/org/apache/camel/component/kafka/consumer/support/KafkaRecordProcessor.java:
##########
@@ -112,30 +115,48 @@ public ProcessingResult processExchange(
             exchange.setException(e);
         }
         if (exchange.getException() != null) {
-            boolean breakOnErrorExit = processException(exchange, partition, lastResult.getPartitionLastOffset(),
+            
+            LOG.debug("an exception was thrown for record at partition {} and offset {}",
+                    record.partition(), record.offset());
+            
+            boolean breakOnErrorExit = processException(exchange, topicPartition, record, lastResult,
                     exceptionHandler);
-            return new ProcessingResult(breakOnErrorExit, lastResult.getPartitionLastOffset(), true);
+            
+            return new ProcessingResult(breakOnErrorExit, lastResult.getPartition(), lastResult.getPartitionLastOffset(), true);
         } else {
-            return new ProcessingResult(false, record.offset(), exchange.getException() != null);
+            return new ProcessingResult(false, record.partition(), record.offset(), exchange.getException() != null);
         }
     }
 
     private boolean processException(
-            Exchange exchange, TopicPartition partition, long partitionLastOffset,
+            Exchange exchange, TopicPartition topicPartition, 
+            ConsumerRecord<Object, Object> record, ProcessingResult lastResult,
             ExceptionHandler exceptionHandler) {
 
         // processing failed due to an unhandled exception, what should we do
         if (configuration.isBreakOnFirstError()) {
+            
+            if (lastResult.getPartition() != -1 &&
+                lastResult.getPartition() != record.partition()) {
+                LOG.error("about to process an exception with UNEXPECTED partition & offset. Got topic partition {}. " + 
+                        " The last result was on partition {} with offset {} but was expecting partition {} with offset {}",
+                        topicPartition.partition(), lastResult.getPartition(), lastResult.getPartitionLastOffset(), 
+                        record.partition(), record.offset());
+            }
+            
             // we are failing and we should break out
             if (LOG.isWarnEnabled()) {
-                LOG.warn("Error during processing {} from topic: {}", exchange, partition.topic(), exchange.getException());
-                LOG.warn("Will seek consumer to offset {} and start polling again.", partitionLastOffset);
+                LOG.warn("Error during processing {} from topic: {}", exchange, topicPartition.topic(), exchange.getException());

Review Comment:
   Obs.: the exception message may also be present elsewhere in the log, but if the intention is to provide more data, I think it would be OK. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-20044 add extra logging for BreakOnFirstError [camel]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11920:
URL: https://github.com/apache/camel/pull/11920#issuecomment-1799335262

   ### Components test results:
   
   | Total | Tested | Failed :x: | Passed :white_check_mark: | 
   | --- | --- | --- |  --- |
   | 2 | 2 | 1 | 1 |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BreakOnFirstError with extra logging [camel]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11920:
URL: https://github.com/apache/camel/pull/11920#issuecomment-1795575851

   :star2: Thank you for your contribution to the Apache Camel project! :star2: 
   
   :warning: Please note that the changes on this PR may be **tested automatically**. 
   
   If necessary Apache Camel Committers may access logs and test results in the job summaries!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-20044 add extra logging for BreakOnFirstError [camel]

Posted by "orpiske (via GitHub)" <gi...@apache.org>.
orpiske commented on code in PR #11920:
URL: https://github.com/apache/camel/pull/11920#discussion_r1384927769


##########
components/camel-kafka/src/main/java/org/apache/camel/component/kafka/consumer/support/KafkaRecordProcessor.java:
##########
@@ -112,30 +115,48 @@ public ProcessingResult processExchange(
             exchange.setException(e);
         }
         if (exchange.getException() != null) {
-            boolean breakOnErrorExit = processException(exchange, partition, lastResult.getPartitionLastOffset(),
+            
+            LOG.debug("an exception was thrown for record at partition {} and offset {}",
+                    record.partition(), record.offset());
+            
+            boolean breakOnErrorExit = processException(exchange, topicPartition, record, lastResult,
                     exceptionHandler);
-            return new ProcessingResult(breakOnErrorExit, lastResult.getPartitionLastOffset(), true);
+            
+            return new ProcessingResult(breakOnErrorExit, lastResult.getPartition(), lastResult.getPartitionLastOffset(), true);
         } else {
-            return new ProcessingResult(false, record.offset(), exchange.getException() != null);
+            return new ProcessingResult(false, record.partition(), record.offset(), exchange.getException() != null);
         }
     }
 
     private boolean processException(
-            Exchange exchange, TopicPartition partition, long partitionLastOffset,
+            Exchange exchange, TopicPartition topicPartition, 
+            ConsumerRecord<Object, Object> record, ProcessingResult lastResult,
             ExceptionHandler exceptionHandler) {
 
         // processing failed due to an unhandled exception, what should we do
         if (configuration.isBreakOnFirstError()) {
+            
+            if (lastResult.getPartition() != -1 &&
+                lastResult.getPartition() != record.partition()) {
+                LOG.error("about to process an exception with UNEXPECTED partition & offset. Got topic partition {}. " + 
+                        " The last result was on partition {} with offset {} but was expecting partition {} with offset {}",
+                        topicPartition.partition(), lastResult.getPartition(), lastResult.getPartitionLastOffset(), 
+                        record.partition(), record.offset());
+            }
+            
             // we are failing and we should break out
             if (LOG.isWarnEnabled()) {
-                LOG.warn("Error during processing {} from topic: {}", exchange, partition.topic(), exchange.getException());
-                LOG.warn("Will seek consumer to offset {} and start polling again.", partitionLastOffset);
+                LOG.warn("Error during processing {} from topic: {}", exchange, topicPartition.topic(), exchange.getException());
+                LOG.warn("Will seek consumer to offset {} on partition {} and start polling again.", 
+                        lastResult.getPartitionLastOffset(), lastResult.getPartition());
             }
 
-            // force commit, so we resume on next poll where we failed except when the failure happened
-            // at the first message in a poll
-            if (partitionLastOffset != AbstractCommitManager.START_OFFSET) {
-                commitManager.forceCommit(partition, partitionLastOffset);
+            // force commit, so we resume on next poll where we failed 
+            // except when the failure happened at the first message in a poll
+            if (lastResult.getPartitionLastOffset() != AbstractCommitManager.START_OFFSET) {
+                // should we use record.offset ?
+                //commitManager.forceCommit(topicPartition, record.offset() - 1);

Review Comment:
   In this case, it's better to remove the code. We can always refer to the git log for details about the history of the code. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BreakOnFirstError with extra logging [camel]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11920:
URL: https://github.com/apache/camel/pull/11920#issuecomment-1798102605

   ### Components test results:
   
   | Total | Tested | Failed :x: | Passed :white_check_mark: | 
   | --- | --- | --- |  --- |
   | 2 | 2 | 1 | 1 |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-20044 add extra logging for BreakOnFirstError [camel]

Posted by "orpiske (via GitHub)" <gi...@apache.org>.
orpiske commented on code in PR #11920:
URL: https://github.com/apache/camel/pull/11920#discussion_r1385250145


##########
components/camel-kafka/src/main/java/org/apache/camel/component/kafka/consumer/support/KafkaRecordProcessor.java:
##########
@@ -112,30 +115,48 @@ public ProcessingResult processExchange(
             exchange.setException(e);
         }
         if (exchange.getException() != null) {
-            boolean breakOnErrorExit = processException(exchange, partition, lastResult.getPartitionLastOffset(),
+            
+            LOG.debug("an exception was thrown for record at partition {} and offset {}",
+                    record.partition(), record.offset());
+            
+            boolean breakOnErrorExit = processException(exchange, topicPartition, record, lastResult,
                     exceptionHandler);
-            return new ProcessingResult(breakOnErrorExit, lastResult.getPartitionLastOffset(), true);
+            
+            return new ProcessingResult(breakOnErrorExit, lastResult.getPartition(), lastResult.getPartitionLastOffset(), true);
         } else {
-            return new ProcessingResult(false, record.offset(), exchange.getException() != null);
+            return new ProcessingResult(false, record.partition(), record.offset(), exchange.getException() != null);
         }
     }
 
     private boolean processException(
-            Exchange exchange, TopicPartition partition, long partitionLastOffset,
+            Exchange exchange, TopicPartition topicPartition, 
+            ConsumerRecord<Object, Object> record, ProcessingResult lastResult,
             ExceptionHandler exceptionHandler) {
 
         // processing failed due to an unhandled exception, what should we do
         if (configuration.isBreakOnFirstError()) {
+            
+            if (lastResult.getPartition() != -1 &&
+                lastResult.getPartition() != record.partition()) {
+                LOG.error("about to process an exception with UNEXPECTED partition & offset. Got topic partition {}. " + 
+                        " The last result was on partition {} with offset {} but was expecting partition {} with offset {}",
+                        topicPartition.partition(), lastResult.getPartition(), lastResult.getPartitionLastOffset(), 
+                        record.partition(), record.offset());
+            }
+            
             // we are failing and we should break out
             if (LOG.isWarnEnabled()) {
-                LOG.warn("Error during processing {} from topic: {}", exchange, partition.topic(), exchange.getException());
-                LOG.warn("Will seek consumer to offset {} and start polling again.", partitionLastOffset);
+                LOG.warn("Error during processing {} from topic: {}", exchange, topicPartition.topic(), exchange.getException());

Review Comment:
   This is the only item remaining. As it is now, it is logging the message. Instead, I would like it to be exactly like this:
   
   ```
   LOG.warn("Error during processing exchange {} from topic {}: {}", exchange, topicPartition.topic(), exchange.getException() ? null : exchange.getException().getMessage(), exchange.getException());
   ```
   
   Just copy/paste the code above and it should be fine ⬆️ 



##########
components/camel-kafka/src/main/java/org/apache/camel/component/kafka/consumer/support/KafkaRecordProcessor.java:
##########
@@ -112,30 +115,48 @@ public ProcessingResult processExchange(
             exchange.setException(e);
         }
         if (exchange.getException() != null) {
-            boolean breakOnErrorExit = processException(exchange, partition, lastResult.getPartitionLastOffset(),
+            
+            LOG.debug("an exception was thrown for record at partition {} and offset {}",
+                    record.partition(), record.offset());
+            
+            boolean breakOnErrorExit = processException(exchange, topicPartition, record, lastResult,
                     exceptionHandler);
-            return new ProcessingResult(breakOnErrorExit, lastResult.getPartitionLastOffset(), true);
+            
+            return new ProcessingResult(breakOnErrorExit, lastResult.getPartition(), lastResult.getPartitionLastOffset(), true);
         } else {
-            return new ProcessingResult(false, record.offset(), exchange.getException() != null);
+            return new ProcessingResult(false, record.partition(), record.offset(), exchange.getException() != null);
         }
     }
 
     private boolean processException(
-            Exchange exchange, TopicPartition partition, long partitionLastOffset,
+            Exchange exchange, TopicPartition topicPartition, 
+            ConsumerRecord<Object, Object> record, ProcessingResult lastResult,
             ExceptionHandler exceptionHandler) {
 
         // processing failed due to an unhandled exception, what should we do
         if (configuration.isBreakOnFirstError()) {
+            
+            if (lastResult.getPartition() != -1 &&
+                lastResult.getPartition() != record.partition()) {
+                LOG.error("about to process an exception with UNEXPECTED partition & offset. Got topic partition {}. " + 
+                        " The last result was on partition {} with offset {} but was expecting partition {} with offset {}",
+                        topicPartition.partition(), lastResult.getPartition(), lastResult.getPartitionLastOffset(), 
+                        record.partition(), record.offset());
+            }
+            
             // we are failing and we should break out
             if (LOG.isWarnEnabled()) {
-                LOG.warn("Error during processing {} from topic: {}", exchange, partition.topic(), exchange.getException());
-                LOG.warn("Will seek consumer to offset {} and start polling again.", partitionLastOffset);
+                LOG.warn("Error during processing {} from topic: {}", exchange, topicPartition.topic(), exchange.getException());

Review Comment:
   This is the only item remaining. As it is now, it is logging the message. Instead, I would like it to be exactly like this:
   
   ```
   LOG.warn("Error during processing exchange {} from topic {}: {}", exchange, topicPartition.topic(), exchange.getException() ? null : exchange.getException().getMessage(), exchange.getException());
   ```
   
   Just copy/paste the code above and it should be fine ⬆️ 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-20044 add extra logging for BreakOnFirstError [camel]

Posted by "orpiske (via GitHub)" <gi...@apache.org>.
orpiske commented on PR #11920:
URL: https://github.com/apache/camel/pull/11920#issuecomment-1802377348

   This needs to be backported to 3.x 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-20044 add extra logging for BreakOnFirstError [camel]

Posted by "CodeSmell (via GitHub)" <gi...@apache.org>.
CodeSmell commented on code in PR #11920:
URL: https://github.com/apache/camel/pull/11920#discussion_r1384905892


##########
components/camel-kafka/src/main/java/org/apache/camel/component/kafka/consumer/support/KafkaRecordProcessor.java:
##########
@@ -112,30 +115,48 @@ public ProcessingResult processExchange(
             exchange.setException(e);
         }
         if (exchange.getException() != null) {
-            boolean breakOnErrorExit = processException(exchange, partition, lastResult.getPartitionLastOffset(),
+            
+            LOG.debug("an exception was thrown for record at partition {} and offset {}",
+                    record.partition(), record.offset());
+            
+            boolean breakOnErrorExit = processException(exchange, topicPartition, record, lastResult,
                     exceptionHandler);
-            return new ProcessingResult(breakOnErrorExit, lastResult.getPartitionLastOffset(), true);
+            
+            return new ProcessingResult(breakOnErrorExit, lastResult.getPartition(), lastResult.getPartitionLastOffset(), true);
         } else {
-            return new ProcessingResult(false, record.offset(), exchange.getException() != null);
+            return new ProcessingResult(false, record.partition(), record.offset(), exchange.getException() != null);
         }
     }
 
     private boolean processException(
-            Exchange exchange, TopicPartition partition, long partitionLastOffset,
+            Exchange exchange, TopicPartition topicPartition, 
+            ConsumerRecord<Object, Object> record, ProcessingResult lastResult,
             ExceptionHandler exceptionHandler) {
 
         // processing failed due to an unhandled exception, what should we do
         if (configuration.isBreakOnFirstError()) {
+            
+            if (lastResult.getPartition() != -1 &&
+                lastResult.getPartition() != record.partition()) {
+                LOG.error("about to process an exception with UNEXPECTED partition & offset. Got topic partition {}. " + 
+                        " The last result was on partition {} with offset {} but was expecting partition {} with offset {}",
+                        topicPartition.partition(), lastResult.getPartition(), lastResult.getPartitionLastOffset(), 
+                        record.partition(), record.offset());
+            }
+            
             // we are failing and we should break out
             if (LOG.isWarnEnabled()) {
-                LOG.warn("Error during processing {} from topic: {}", exchange, partition.topic(), exchange.getException());
-                LOG.warn("Will seek consumer to offset {} and start polling again.", partitionLastOffset);
+                LOG.warn("Error during processing {} from topic: {}", exchange, topicPartition.topic(), exchange.getException());
+                LOG.warn("Will seek consumer to offset {} on partition {} and start polling again.", 
+                        lastResult.getPartitionLastOffset(), lastResult.getPartition());
             }
 
-            // force commit, so we resume on next poll where we failed except when the failure happened
-            // at the first message in a poll
-            if (partitionLastOffset != AbstractCommitManager.START_OFFSET) {
-                commitManager.forceCommit(partition, partitionLastOffset);
+            // force commit, so we resume on next poll where we failed 
+            // except when the failure happened at the first message in a poll
+            if (lastResult.getPartitionLastOffset() != AbstractCommitManager.START_OFFSET) {
+                // should we use record.offset ?
+                //commitManager.forceCommit(topicPartition, record.offset() - 1);

Review Comment:
   this change that is commented out is what mostly addresses the issues found in CAMEL-20044 and CAMEL-19894
   
   I can submit this as a separate PR 
   I commented it out here b/c I had not added unit tests and b/c I wasn't sure if that would impact other ways that the Camel riders would want breakOnFirstError to work
   
   I did see CAMEL-20089 but hope this fix can move forward as we it does fix 2 other issues 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-20044 add extra logging for BreakOnFirstError [camel]

Posted by "CodeSmell (via GitHub)" <gi...@apache.org>.
CodeSmell commented on code in PR #11920:
URL: https://github.com/apache/camel/pull/11920#discussion_r1384954679


##########
components/camel-kafka/src/main/java/org/apache/camel/component/kafka/consumer/support/KafkaRecordProcessor.java:
##########
@@ -112,30 +115,48 @@ public ProcessingResult processExchange(
             exchange.setException(e);
         }
         if (exchange.getException() != null) {
-            boolean breakOnErrorExit = processException(exchange, partition, lastResult.getPartitionLastOffset(),
+            
+            LOG.debug("an exception was thrown for record at partition {} and offset {}",
+                    record.partition(), record.offset());
+            
+            boolean breakOnErrorExit = processException(exchange, topicPartition, record, lastResult,
                     exceptionHandler);
-            return new ProcessingResult(breakOnErrorExit, lastResult.getPartitionLastOffset(), true);
+            
+            return new ProcessingResult(breakOnErrorExit, lastResult.getPartition(), lastResult.getPartitionLastOffset(), true);
         } else {
-            return new ProcessingResult(false, record.offset(), exchange.getException() != null);
+            return new ProcessingResult(false, record.partition(), record.offset(), exchange.getException() != null);
         }
     }
 
     private boolean processException(
-            Exchange exchange, TopicPartition partition, long partitionLastOffset,
+            Exchange exchange, TopicPartition topicPartition, 
+            ConsumerRecord<Object, Object> record, ProcessingResult lastResult,
             ExceptionHandler exceptionHandler) {
 
         // processing failed due to an unhandled exception, what should we do
         if (configuration.isBreakOnFirstError()) {
+            
+            if (lastResult.getPartition() != -1 &&
+                lastResult.getPartition() != record.partition()) {
+                LOG.error("about to process an exception with UNEXPECTED partition & offset. Got topic partition {}. " + 
+                        " The last result was on partition {} with offset {} but was expecting partition {} with offset {}",
+                        topicPartition.partition(), lastResult.getPartition(), lastResult.getPartitionLastOffset(), 
+                        record.partition(), record.offset());
+            }
+            
             // we are failing and we should break out
             if (LOG.isWarnEnabled()) {
-                LOG.warn("Error during processing {} from topic: {}", exchange, partition.topic(), exchange.getException());
-                LOG.warn("Will seek consumer to offset {} and start polling again.", partitionLastOffset);
+                LOG.warn("Error during processing {} from topic: {}", exchange, topicPartition.topic(), exchange.getException());
+                LOG.warn("Will seek consumer to offset {} on partition {} and start polling again.", 
+                        lastResult.getPartitionLastOffset(), lastResult.getPartition());
             }
 
-            // force commit, so we resume on next poll where we failed except when the failure happened
-            // at the first message in a poll
-            if (partitionLastOffset != AbstractCommitManager.START_OFFSET) {
-                commitManager.forceCommit(partition, partitionLastOffset);
+            // force commit, so we resume on next poll where we failed 
+            // except when the failure happened at the first message in a poll
+            if (lastResult.getPartitionLastOffset() != AbstractCommitManager.START_OFFSET) {
+                // should we use record.offset ?
+                //commitManager.forceCommit(topicPartition, record.offset() - 1);

Review Comment:
   I am working on a separate PR for the fix
   Is this worth pursuing?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-20044 add extra logging for BreakOnFirstError [camel]

Posted by "orpiske (via GitHub)" <gi...@apache.org>.
orpiske commented on code in PR #11920:
URL: https://github.com/apache/camel/pull/11920#discussion_r1384959149


##########
components/camel-kafka/src/main/java/org/apache/camel/component/kafka/consumer/support/KafkaRecordProcessor.java:
##########
@@ -112,30 +115,48 @@ public ProcessingResult processExchange(
             exchange.setException(e);
         }
         if (exchange.getException() != null) {
-            boolean breakOnErrorExit = processException(exchange, partition, lastResult.getPartitionLastOffset(),
+            
+            LOG.debug("an exception was thrown for record at partition {} and offset {}",
+                    record.partition(), record.offset());
+            
+            boolean breakOnErrorExit = processException(exchange, topicPartition, record, lastResult,
                     exceptionHandler);
-            return new ProcessingResult(breakOnErrorExit, lastResult.getPartitionLastOffset(), true);
+            
+            return new ProcessingResult(breakOnErrorExit, lastResult.getPartition(), lastResult.getPartitionLastOffset(), true);
         } else {
-            return new ProcessingResult(false, record.offset(), exchange.getException() != null);
+            return new ProcessingResult(false, record.partition(), record.offset(), exchange.getException() != null);
         }
     }
 
     private boolean processException(
-            Exchange exchange, TopicPartition partition, long partitionLastOffset,
+            Exchange exchange, TopicPartition topicPartition, 
+            ConsumerRecord<Object, Object> record, ProcessingResult lastResult,
             ExceptionHandler exceptionHandler) {
 
         // processing failed due to an unhandled exception, what should we do
         if (configuration.isBreakOnFirstError()) {
+            
+            if (lastResult.getPartition() != -1 &&
+                lastResult.getPartition() != record.partition()) {
+                LOG.error("about to process an exception with UNEXPECTED partition & offset. Got topic partition {}. " + 
+                        " The last result was on partition {} with offset {} but was expecting partition {} with offset {}",
+                        topicPartition.partition(), lastResult.getPartition(), lastResult.getPartitionLastOffset(), 
+                        record.partition(), record.offset());
+            }
+            
             // we are failing and we should break out
             if (LOG.isWarnEnabled()) {
-                LOG.warn("Error during processing {} from topic: {}", exchange, partition.topic(), exchange.getException());
-                LOG.warn("Will seek consumer to offset {} and start polling again.", partitionLastOffset);
+                LOG.warn("Error during processing {} from topic: {}", exchange, topicPartition.topic(), exchange.getException());
+                LOG.warn("Will seek consumer to offset {} on partition {} and start polling again.", 
+                        lastResult.getPartitionLastOffset(), lastResult.getPartition());
             }
 
-            // force commit, so we resume on next poll where we failed except when the failure happened
-            // at the first message in a poll
-            if (partitionLastOffset != AbstractCommitManager.START_OFFSET) {
-                commitManager.forceCommit(partition, partitionLastOffset);
+            // force commit, so we resume on next poll where we failed 
+            // except when the failure happened at the first message in a poll
+            if (lastResult.getPartitionLastOffset() != AbstractCommitManager.START_OFFSET) {
+                // should we use record.offset ?
+                //commitManager.forceCommit(topicPartition, record.offset() - 1);

Review Comment:
   Yes it is. If you can fix it, we'll be happy to look at your proposed changes. 
   
   Obs.: I am talking about being better to remove the commented code. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-20044 add extra logging for BreakOnFirstError [camel]

Posted by "CodeSmell (via GitHub)" <gi...@apache.org>.
CodeSmell commented on code in PR #11920:
URL: https://github.com/apache/camel/pull/11920#discussion_r1384965058


##########
components/camel-kafka/src/main/java/org/apache/camel/component/kafka/consumer/support/KafkaRecordProcessor.java:
##########
@@ -112,30 +115,48 @@ public ProcessingResult processExchange(
             exchange.setException(e);
         }
         if (exchange.getException() != null) {
-            boolean breakOnErrorExit = processException(exchange, partition, lastResult.getPartitionLastOffset(),
+            
+            LOG.debug("an exception was thrown for record at partition {} and offset {}",
+                    record.partition(), record.offset());
+            
+            boolean breakOnErrorExit = processException(exchange, topicPartition, record, lastResult,
                     exceptionHandler);
-            return new ProcessingResult(breakOnErrorExit, lastResult.getPartitionLastOffset(), true);
+            
+            return new ProcessingResult(breakOnErrorExit, lastResult.getPartition(), lastResult.getPartitionLastOffset(), true);
         } else {
-            return new ProcessingResult(false, record.offset(), exchange.getException() != null);
+            return new ProcessingResult(false, record.partition(), record.offset(), exchange.getException() != null);
         }
     }
 
     private boolean processException(
-            Exchange exchange, TopicPartition partition, long partitionLastOffset,
+            Exchange exchange, TopicPartition topicPartition, 
+            ConsumerRecord<Object, Object> record, ProcessingResult lastResult,
             ExceptionHandler exceptionHandler) {
 
         // processing failed due to an unhandled exception, what should we do
         if (configuration.isBreakOnFirstError()) {
+            
+            if (lastResult.getPartition() != -1 &&
+                lastResult.getPartition() != record.partition()) {
+                LOG.error("about to process an exception with UNEXPECTED partition & offset. Got topic partition {}. " + 
+                        " The last result was on partition {} with offset {} but was expecting partition {} with offset {}",
+                        topicPartition.partition(), lastResult.getPartition(), lastResult.getPartitionLastOffset(), 
+                        record.partition(), record.offset());
+            }
+            
             // we are failing and we should break out
             if (LOG.isWarnEnabled()) {
-                LOG.warn("Error during processing {} from topic: {}", exchange, partition.topic(), exchange.getException());
-                LOG.warn("Will seek consumer to offset {} and start polling again.", partitionLastOffset);
+                LOG.warn("Error during processing {} from topic: {}", exchange, topicPartition.topic(), exchange.getException());
+                LOG.warn("Will seek consumer to offset {} on partition {} and start polling again.", 
+                        lastResult.getPartitionLastOffset(), lastResult.getPartition());
             }
 
-            // force commit, so we resume on next poll where we failed except when the failure happened
-            // at the first message in a poll
-            if (partitionLastOffset != AbstractCommitManager.START_OFFSET) {
-                commitManager.forceCommit(partition, partitionLastOffset);
+            // force commit, so we resume on next poll where we failed 
+            // except when the failure happened at the first message in a poll
+            if (lastResult.getPartitionLastOffset() != AbstractCommitManager.START_OFFSET) {
+                // should we use record.offset ?
+                //commitManager.forceCommit(topicPartition, record.offset() - 1);

Review Comment:
   Cool. Removed



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] BreakOnFirstError with extra logging [camel]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11920:
URL: https://github.com/apache/camel/pull/11920#issuecomment-1798102641

   :no_entry_sign: There are (likely) no changes in core core to be tested in this PR


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-20044 add extra logging for BreakOnFirstError [camel]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11920:
URL: https://github.com/apache/camel/pull/11920#issuecomment-1799303223

   ### Components test results:
   
   | Total | Tested | Failed :x: | Passed :white_check_mark: | 
   | --- | --- | --- |  --- |
   | 2 | 2 | 1 | 1 |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-20044 add extra logging for BreakOnFirstError [camel]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11920:
URL: https://github.com/apache/camel/pull/11920#issuecomment-1802115592

   :no_entry_sign: There are (likely) no changes in core core to be tested in this PR


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org