[GitHub] [hudi] YuweiXiao commented on a change in pull request #4222: [HUDI-2849] improve SparkUI job description for write path

[GitBox <gi...@apache.org>]

YuweiXiao commented on a change in pull request #4222:
URL: https://github.com/apache/hudi/pull/4222#discussion_r764627374



##########
File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/DeltaSync.java
##########
@@ -441,9 +441,10 @@ public void refreshTimeline() throws IOException {
       return null;
     }
 
+    jssc.setJobGroup(this.getClass().getSimpleName(), "Checking if input is empty");
     if ((!avroRDDOptional.isPresent()) || (avroRDDOptional.get().isEmpty())) {
       LOG.info("No new data, perform empty commit.");
-      return Pair.of(schemaProvider, Pair.of(checkpointStr, jssc.emptyRDD()));
+      return Pair.of(schemaProvider, Pair.of(checkpointStr, null));

Review comment:
       It saves one call to isEmpty() in `DeltaSync::writeToSink`, which could further eliminate one spark job.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org