You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2021/08/24 07:19:39 UTC

[GitHub] [hudi] yihua opened a new pull request #3529: [HUDI-2351] Extract common FS and IO utils for marker mechanism

yihua opened a new pull request #3529:
URL: https://github.com/apache/hudi/pull/3529


   ## What is the purpose of the pull request
   
   This PR extracts the common FS and IO util methods used by marker-related operations. 
   
   ## Brief change log
   
   - Adds new methods in `FSUtils` and `FileIOUtils`
     - `parallelizeSubPathProcess()`: a general method for going through sub paths in parallel using `HoodieEngineContext` with custom predicates and processing logic for each sub path.
     - `deleteDir()`: delete a directory with parallelism
     - `readAsUTFStringLines()`: read file content into lines
   - Uses the above methods in marker mechanisms wherever possible.
   
   ## Verify this pull request
   
   - Unit tests around `WriteMarkers` succeed
   - Manually runs spark jobs of bulk inserts with `direct` and `timeline-server-based` marker types.  Both of them succeed locally.
   
   ## Committer checklist
   
    - [ ] Has a corresponding JIRA in PR title & commit
    
    - [ ] Commit message is descriptive of the change
    
    - [ ] CI is green
   
    - [ ] Necessary doc changes done or have another open PR
          
    - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hudi] hudi-bot edited a comment on pull request #3529: [HUDI-2351] Extract common FS and IO utils for marker mechanism

Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3529:
URL: https://github.com/apache/hudi/pull/3529#issuecomment-904389285


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "3a4a8c779f412ffab9216b84263d124da28cb672",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1871",
       "triggerID" : "3a4a8c779f412ffab9216b84263d124da28cb672",
       "triggerType" : "PUSH"
     }, {
       "hash" : "90e5845e8dc9ad66ac63bb7fc5605ffd4be4eef9",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2083",
       "triggerID" : "90e5845e8dc9ad66ac63bb7fc5605ffd4be4eef9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "89fc744f269cce62e8ff2562c1e9c6e32dadbc98",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "89fc744f269cce62e8ff2562c1e9c6e32dadbc98",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 3a4a8c779f412ffab9216b84263d124da28cb672 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1871) 
   * 90e5845e8dc9ad66ac63bb7fc5605ffd4be4eef9 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2083) 
   * 89fc744f269cce62e8ff2562c1e9c6e32dadbc98 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run travis` re-run the last Travis build
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hudi] hudi-bot edited a comment on pull request #3529: [HUDI-2351] Extract common FS and IO utils for marker mechanism

Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3529:
URL: https://github.com/apache/hudi/pull/3529#issuecomment-904389285


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "3a4a8c779f412ffab9216b84263d124da28cb672",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1871",
       "triggerID" : "3a4a8c779f412ffab9216b84263d124da28cb672",
       "triggerType" : "PUSH"
     }, {
       "hash" : "90e5845e8dc9ad66ac63bb7fc5605ffd4be4eef9",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2083",
       "triggerID" : "90e5845e8dc9ad66ac63bb7fc5605ffd4be4eef9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "89fc744f269cce62e8ff2562c1e9c6e32dadbc98",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2088",
       "triggerID" : "89fc744f269cce62e8ff2562c1e9c6e32dadbc98",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 90e5845e8dc9ad66ac63bb7fc5605ffd4be4eef9 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2083) 
   * 89fc744f269cce62e8ff2562c1e9c6e32dadbc98 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2088) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run travis` re-run the last Travis build
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hudi] hudi-bot commented on pull request #3529: [HUDI-2351] Extract common FS and IO utils for marker mechanism

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #3529:
URL: https://github.com/apache/hudi/pull/3529#issuecomment-904389285


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "3a4a8c779f412ffab9216b84263d124da28cb672",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3a4a8c779f412ffab9216b84263d124da28cb672",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 3a4a8c779f412ffab9216b84263d124da28cb672 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run travis` re-run the last Travis build
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hudi] yihua commented on a change in pull request #3529: [HUDI-2351] Extract common FS and IO utils for marker mechanism

Posted by GitBox <gi...@apache.org>.
yihua commented on a change in pull request #3529:
URL: https://github.com/apache/hudi/pull/3529#discussion_r703579575



##########
File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/upgrade/BaseTwoToOneDowngradeHandler.java
##########
@@ -117,26 +117,26 @@ private void convertToDirectMarkers(final String commitInstantTime,
     } else {
       // In case of partial failures during downgrade, there is a chance that marker type file was deleted,
       // but timeline server based marker files are left.  So deletes them if any
-      deleteTimelineBasedMarkerFiles(markerDir, fileSystem);
+      deleteTimelineBasedMarkerFiles(
+          context, markerDir, fileSystem, table.getConfig().getMarkersDeleteParallelism());
     }
   }
 
-  private void deleteTimelineBasedMarkerFiles(String markerDir, FileSystem fileSystem) throws IOException {
+  private void deleteTimelineBasedMarkerFiles(HoodieEngineContext context, String markerDir,
+                                              FileSystem fileSystem, int parallelism) throws IOException {
     // Deletes timeline based marker files if any.
-    Path dirPath = new Path(markerDir);
-    FileStatus[] fileStatuses = fileSystem.listStatus(dirPath);
     Predicate<FileStatus> prefixFilter = fileStatus ->
         fileStatus.getPath().getName().startsWith(MARKERS_FILENAME_PREFIX);
-    List<String> markerDirSubPaths = Arrays.stream(fileStatuses)
-        .filter(prefixFilter)
-        .map(fileStatus -> fileStatus.getPath().toString())
-        .collect(Collectors.toList());
-    markerDirSubPaths.forEach(fileToDelete -> {
-      try {
-        fileSystem.delete(new Path(fileToDelete), false);
-      } catch (IOException e) {
-        Log.warn("Deleting Timeline based marker files failed ", e);
-      }
-    });
+    FSUtils.parallelizeSubPathProcess(context, fileSystem, new Path(markerDir), parallelism,
+        prefixFilter, pairOfSubPathAndConf -> {

Review comment:
       Good Catch.  Fixed.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hudi] hudi-bot edited a comment on pull request #3529: [HUDI-2351] Extract common FS and IO utils for marker mechanism

Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3529:
URL: https://github.com/apache/hudi/pull/3529#issuecomment-904389285


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "3a4a8c779f412ffab9216b84263d124da28cb672",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1871",
       "triggerID" : "3a4a8c779f412ffab9216b84263d124da28cb672",
       "triggerType" : "PUSH"
     }, {
       "hash" : "90e5845e8dc9ad66ac63bb7fc5605ffd4be4eef9",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2083",
       "triggerID" : "90e5845e8dc9ad66ac63bb7fc5605ffd4be4eef9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "89fc744f269cce62e8ff2562c1e9c6e32dadbc98",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "89fc744f269cce62e8ff2562c1e9c6e32dadbc98",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 90e5845e8dc9ad66ac63bb7fc5605ffd4be4eef9 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2083) 
   * 89fc744f269cce62e8ff2562c1e9c6e32dadbc98 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run travis` re-run the last Travis build
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hudi] nsivabalan commented on pull request #3529: [HUDI-2351] Extract common FS and IO utils for marker mechanism

Posted by GitBox <gi...@apache.org>.
nsivabalan commented on pull request #3529:
URL: https://github.com/apache/hudi/pull/3529#issuecomment-906450087


   Can we please add unit tests for new util methods? 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hudi] nsivabalan merged pull request #3529: [HUDI-2351] Extract common FS and IO utils for marker mechanism

Posted by GitBox <gi...@apache.org>.
nsivabalan merged pull request #3529:
URL: https://github.com/apache/hudi/pull/3529


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hudi] nsivabalan commented on a change in pull request #3529: [HUDI-2351] Extract common FS and IO utils for marker mechanism

Posted by GitBox <gi...@apache.org>.
nsivabalan commented on a change in pull request #3529:
URL: https://github.com/apache/hudi/pull/3529#discussion_r696668520



##########
File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/upgrade/BaseTwoToOneDowngradeHandler.java
##########
@@ -117,26 +117,26 @@ private void convertToDirectMarkers(final String commitInstantTime,
     } else {
       // In case of partial failures during downgrade, there is a chance that marker type file was deleted,
       // but timeline server based marker files are left.  So deletes them if any
-      deleteTimelineBasedMarkerFiles(markerDir, fileSystem);
+      deleteTimelineBasedMarkerFiles(
+          context, markerDir, fileSystem, table.getConfig().getMarkersDeleteParallelism());
     }
   }
 
-  private void deleteTimelineBasedMarkerFiles(String markerDir, FileSystem fileSystem) throws IOException {
+  private void deleteTimelineBasedMarkerFiles(HoodieEngineContext context, String markerDir,
+                                              FileSystem fileSystem, int parallelism) throws IOException {
     // Deletes timeline based marker files if any.
-    Path dirPath = new Path(markerDir);
-    FileStatus[] fileStatuses = fileSystem.listStatus(dirPath);
     Predicate<FileStatus> prefixFilter = fileStatus ->
         fileStatus.getPath().getName().startsWith(MARKERS_FILENAME_PREFIX);
-    List<String> markerDirSubPaths = Arrays.stream(fileStatuses)
-        .filter(prefixFilter)
-        .map(fileStatus -> fileStatus.getPath().toString())
-        .collect(Collectors.toList());
-    markerDirSubPaths.forEach(fileToDelete -> {
-      try {
-        fileSystem.delete(new Path(fileToDelete), false);
-      } catch (IOException e) {
-        Log.warn("Deleting Timeline based marker files failed ", e);
-      }
-    });
+    FSUtils.parallelizeSubPathProcess(context, fileSystem, new Path(markerDir), parallelism,
+        prefixFilter, pairOfSubPathAndConf -> {

Review comment:
       sorry, shouldn't we be calling the new util method
   ```
   pairOfSubPathAndConf -> deleteSubPath(pairOfSubPathAndConf.getKey(), pairOfSubPathAndConf.getValue())
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hudi] hudi-bot edited a comment on pull request #3529: [HUDI-2351] Extract common FS and IO utils for marker mechanism

Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3529:
URL: https://github.com/apache/hudi/pull/3529#issuecomment-904389285


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "3a4a8c779f412ffab9216b84263d124da28cb672",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1871",
       "triggerID" : "3a4a8c779f412ffab9216b84263d124da28cb672",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 3a4a8c779f412ffab9216b84263d124da28cb672 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1871) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run travis` re-run the last Travis build
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hudi] hudi-bot edited a comment on pull request #3529: [HUDI-2351] Extract common FS and IO utils for marker mechanism

Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3529:
URL: https://github.com/apache/hudi/pull/3529#issuecomment-904389285


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "3a4a8c779f412ffab9216b84263d124da28cb672",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1871",
       "triggerID" : "3a4a8c779f412ffab9216b84263d124da28cb672",
       "triggerType" : "PUSH"
     }, {
       "hash" : "90e5845e8dc9ad66ac63bb7fc5605ffd4be4eef9",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2083",
       "triggerID" : "90e5845e8dc9ad66ac63bb7fc5605ffd4be4eef9",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 3a4a8c779f412ffab9216b84263d124da28cb672 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1871) 
   * 90e5845e8dc9ad66ac63bb7fc5605ffd4be4eef9 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2083) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run travis` re-run the last Travis build
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hudi] hudi-bot edited a comment on pull request #3529: [HUDI-2351] Extract common FS and IO utils for marker mechanism

Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3529:
URL: https://github.com/apache/hudi/pull/3529#issuecomment-904389285


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "3a4a8c779f412ffab9216b84263d124da28cb672",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1871",
       "triggerID" : "3a4a8c779f412ffab9216b84263d124da28cb672",
       "triggerType" : "PUSH"
     }, {
       "hash" : "90e5845e8dc9ad66ac63bb7fc5605ffd4be4eef9",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "90e5845e8dc9ad66ac63bb7fc5605ffd4be4eef9",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 3a4a8c779f412ffab9216b84263d124da28cb672 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1871) 
   * 90e5845e8dc9ad66ac63bb7fc5605ffd4be4eef9 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run travis` re-run the last Travis build
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hudi] hudi-bot edited a comment on pull request #3529: [HUDI-2351] Extract common FS and IO utils for marker mechanism

Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3529:
URL: https://github.com/apache/hudi/pull/3529#issuecomment-904389285


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "3a4a8c779f412ffab9216b84263d124da28cb672",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1871",
       "triggerID" : "3a4a8c779f412ffab9216b84263d124da28cb672",
       "triggerType" : "PUSH"
     }, {
       "hash" : "90e5845e8dc9ad66ac63bb7fc5605ffd4be4eef9",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2083",
       "triggerID" : "90e5845e8dc9ad66ac63bb7fc5605ffd4be4eef9",
       "triggerType" : "PUSH"
     }, {
       "hash" : "89fc744f269cce62e8ff2562c1e9c6e32dadbc98",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2088",
       "triggerID" : "89fc744f269cce62e8ff2562c1e9c6e32dadbc98",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 89fc744f269cce62e8ff2562c1e9c6e32dadbc98 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2088) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run travis` re-run the last Travis build
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hudi] hudi-bot edited a comment on pull request #3529: [HUDI-2351] Extract common FS and IO utils for marker mechanism

Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3529:
URL: https://github.com/apache/hudi/pull/3529#issuecomment-904389285


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "3a4a8c779f412ffab9216b84263d124da28cb672",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1871",
       "triggerID" : "3a4a8c779f412ffab9216b84263d124da28cb672",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 3a4a8c779f412ffab9216b84263d124da28cb672 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1871) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run travis` re-run the last Travis build
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org