You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by GitBox <gi...@apache.org> on 2020/02/13 00:32:44 UTC
[GitHub] [samza] bkonold commented on a change in pull request #1236:
SAMZA-2419: removing duplicate cleaning of stores
bkonold commented on a change in pull request #1236: SAMZA-2419: removing duplicate cleaning of stores
URL: https://github.com/apache/samza/pull/1236#discussion_r378590970
##########
File path: samza-core/src/main/java/org/apache/samza/storage/TransactionalStateTaskRestoreManager.java
##########
@@ -243,30 +243,19 @@ static StoreActions getStoreActions(
timeSinceLastCheckpointInMs = System.currentTimeMillis() -
checkpointedChangelogOffset.getCheckpointId().getMillis();
}
-
- // if the clean.store.start config is set, delete the currentDir, restore from oldest offset to checkpointed
- if (storageEngine.getStoreProperties().isPersistedToDisk() && new StorageConfig(
- config).getCleanLoggedStoreDirsOnStart(storeName)) {
- File currentDir = storageManagerUtil.getTaskStoreDir(nonLoggedStoreBaseDirectory, storeName, taskName, taskMode);
- LOG.info("Marking current directory: {} for store: {} in task: {}.", currentDir, storeName, taskName);
- storeDirsToDelete.put(storeName, currentDir);
- LOG.info("Marking restore offsets for store: {} in task: {} to {}, {} ", storeName, taskName, oldestOffset, checkpointedOffset);
- storesToRestore.put(storeName, new RestoreOffsets(oldestOffset, checkpointedOffset));
- return;
- }
// if the clean.store.start config is set, delete the currentDir, restore from oldest offset to checkpointed
if (storageEngine.getStoreProperties().isPersistedToDisk() && new StorageConfig(
Review comment:
to recap: @mynameborat 's question was with regard to side inputs, which are persisted and non-logged but are not to be deleted. this is a non-issue since side inputs are not hooked in through TransactionalStateTaskRestoreManager but instead TaskSideInputStorageManager. @mynameborat feel to comment if i am misunderstanding our discussion.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services