You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by GitBox <gi...@apache.org> on 2020/03/27 07:21:00 UTC
[GitHub] [samza] PanTheMan opened a new pull request #1333: SAMZA-2497:
Close MetadataResourceUtil when job is stopped
PanTheMan opened a new pull request #1333: SAMZA-2497: Close MetadataResourceUtil when job is stopped
URL: https://github.com/apache/samza/pull/1333
**Symptoms**: Error message about a Kafka consumer allocated but not closed appears in our log messages
**Cause**: The MetadataResouceUtil has a Kafka checkpoint manager that is never closed in our code. A LifeCycleAwareConsumer sees this and complains about it.
**Fix:**: Add a stop in MetadataResourceUtil that will close the checkpoint manager in it. Then have MetadataResourceUtil.stop called everywhere it's used when the job is stopped
**Tests**: Any job that uses ProcessJob or PassthroughJobCoordinator will see errors in the log files about a Kafka consumer allocated but not closed. With this fix in, that error shouldn't appear
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [samza] PanTheMan commented on issue #1333: SAMZA-2497: Close
MetadataResourceUtil when job is stopped
Posted by GitBox <gi...@apache.org>.
PanTheMan commented on issue #1333: SAMZA-2497: Close MetadataResourceUtil when job is stopped
URL: https://github.com/apache/samza/pull/1333#issuecomment-604852630
@xinyuiscool if you could review this rb
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [samza] Sanil15 commented on a change in pull request #1333:
SAMZA-2497: Close MetadataResourceUtil when job is stopped
Posted by GitBox <gi...@apache.org>.
Sanil15 commented on a change in pull request #1333: SAMZA-2497: Close MetadataResourceUtil when job is stopped
URL: https://github.com/apache/samza/pull/1333#discussion_r405025357
##########
File path: samza-core/src/main/java/org/apache/samza/zk/ZkJobCoordinator.java
##########
@@ -312,7 +313,7 @@ JobModel readJobModelFromMetadataStore(String zkJobModelVersion) {
@VisibleForTesting
void loadMetadataResources(JobModel jobModel) {
try {
- MetadataResourceUtil metadataResourceUtil = createMetadataResourceUtil(jobModel, config);
+ metadataResourceUtil = createMetadataResourceUtil(jobModel, config);
Review comment:
Where is `metadataResourceUtil#stop` is being stopped for ZkJobCoordinator?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services