[GitHub] [hive] pvary commented on a change in pull request #690: HIVE-21910: Multiple target location generation in HostAffinitySplitLocationProvider

[GitBox <gi...@apache.org>]

pvary commented on a change in pull request #690: HIVE-21910: Multiple target location generation in HostAffinitySplitLocationProvider
URL: https://github.com/apache/hive/pull/690#discussion_r298556011
 
 

 ##########
 File path: ql/src/java/org/apache/hadoop/hive/ql/exec/tez/HostAffinitySplitLocationProvider.java
 ##########
 @@ -72,11 +78,17 @@ public HostAffinitySplitLocationProvider(List<String> knownLocations) {
     FileSplit fsplit = (FileSplit) split;
     String splitDesc = "Split at " + fsplit.getPath() + " with offset= " + fsplit.getStart()
         + ", length=" + fsplit.getLength();
-    List<String> preferredLocations = preferLocations(fsplit);
-    String location =
-        preferredLocations.get(determineLocation(preferredLocations, fsplit.getPath().toString(),
-            fsplit.getStart(), splitDesc));
-    return (location != null) ? new String[] { location } : null;
+    List<String> preferredLocations = new ArrayList<>(preferLocations(fsplit));
 
 Review comment:
   We might want to keep it configurable.
   Until the cluster reaches the threshold where the highest loaded node starts to struggle for resources keeping the tasks aligned with HDFS location still makes sense.
   What do you think?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org