You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2021/02/22 22:38:53 UTC

[GitHub] [kafka] wcarlson5 opened a new pull request #10180: KAFKA- 12347: expose offsets to streams client

wcarlson5 opened a new pull request #10180:
URL: https://github.com/apache/kafka/pull/10180


   collect the offsets after they are committed
   
   ### Committer Checklist (excluded from commit message)
   - [ ] Verify design and implementation 
   - [ ] Verify test coverage and CI build status
   - [ ] Verify documentation (including upgrade notes)
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] mjsax commented on a change in pull request #10180: KAFKA- 12347: expose offsets to streams client (WIP)

Posted by GitBox <gi...@apache.org>.
mjsax commented on a change in pull request #10180:
URL: https://github.com/apache/kafka/pull/10180#discussion_r580731514



##########
File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java
##########
@@ -91,6 +91,7 @@
     // includes assigned & initialized tasks and unassigned tasks we locked temporarily during rebalance
     private final Set<TaskId> lockedTaskDirectories = new HashSet<>();
     private java.util.function.Consumer<Set<TopicPartition>> resetter;
+    private Map<TopicPartition, Long> committedOffsets = new HashMap<>();

Review comment:
       During a rebalance, we should delete all entries for partitions we don't own any longer.
   
   Should we also pre-populate this map when we init a task (cf `StreamsTask#initializeMetadata()`)




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] wcarlson5 closed pull request #10180: KAFKA- 12347: expose offsets to streams client (WIP)

Posted by GitBox <gi...@apache.org>.
wcarlson5 closed pull request #10180:
URL: https://github.com/apache/kafka/pull/10180


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] mjsax commented on a change in pull request #10180: KAFKA- 12347: expose offsets to streams client (WIP)

Posted by GitBox <gi...@apache.org>.
mjsax commented on a change in pull request #10180:
URL: https://github.com/apache/kafka/pull/10180#discussion_r580731029



##########
File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java
##########
@@ -307,6 +308,9 @@ public boolean isRunning() {
     private final ProcessingMode processingMode;
     private AtomicBoolean leaveGroupRequested;
 
+    private final Map<TopicPartition, Integer> committedOffsets;

Review comment:
       Seems to be unused?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] wcarlson5 commented on pull request #10180: KAFKA- 12347: expose offsets to streams client (WIP)

Posted by GitBox <gi...@apache.org>.
wcarlson5 commented on pull request #10180:
URL: https://github.com/apache/kafka/pull/10180#issuecomment-786196837


   Is moved to https://github.com/apache/kafka/pull/10211


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org