You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2020/04/16 00:19:16 UTC
[GitHub] [incubator-hudi] afilipchik commented on a change in pull request
#1515: [HUDI-795] Ignoring missing aux folder
afilipchik commented on a change in pull request #1515: [HUDI-795] Ignoring missing aux folder
URL: https://github.com/apache/incubator-hudi/pull/1515#discussion_r409208032
##########
File path: hudi-client/src/main/java/org/apache/hudi/table/HoodieCommitArchiveLog.java
##########
@@ -219,14 +220,23 @@ private boolean deleteArchivedInstants(List<HoodieInstant> archivedInstants) thr
* @throws IOException in case of error
*/
private boolean deleteAllInstantsOlderorEqualsInAuxMetaFolder(HoodieInstant thresholdInstant) throws IOException {
- List<HoodieInstant> instants = metaClient.scanHoodieInstantsFromFileSystem(
- new Path(metaClient.getMetaAuxiliaryPath()), HoodieActiveTimeline.VALID_EXTENSIONS_IN_ACTIVE_TIMELINE, false);
+ List<HoodieInstant> instants = null;
+ boolean success = true;
+ try {
+ instants =
+ metaClient.scanHoodieInstantsFromFileSystem(
+ new Path(metaClient.getMetaAuxiliaryPath()),
+ HoodieActiveTimeline.VALID_EXTENSIONS_IN_ACTIVE_TIMELINE,
+ false);
+ } catch (FileNotFoundException e) {
Review comment:
not sure about it. Did see that folder disappears/reappears sometimes. Can be related to the way GCS connector treats a situation when all the files from the folder are removed.
On gcs folders are not real, they are just logical. So, file with name: gs://bucket/folder/file doesn't live in the folder "folder", it just has a long "/" separated name. I saw some logic in the GCS FS java implementation that creates fake file "/" when everything is removed from the logical folder. So, my speculation is that if this call fails or not enabled, that removing all the files from the folder will end up if folder being removed as well.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services