[GitHub] [kylin] shaofengshi commented on a change in pull request #1173: KYLIN-4385 HiveSink support object storage

[GitBox <gi...@apache.org>]

shaofengshi commented on a change in pull request #1173:
URL: https://github.com/apache/kylin/pull/1173#discussion_r415068769



##########
File path: metrics-reporter-hive/src/main/java/org/apache/kylin/metrics/lib/impl/hive/HiveProducer.java
##########
@@ -97,7 +105,10 @@ public void onRemoval(RemovalNotification<Pair<String, String>, Pair<String, Lis
         } catch (UnknownHostException e) {
             hostName = "UNKNOWN";
         }
-        CONTENT_FILE_NAME = hostName + "-part-0000";
+        CONTENT_FILE_NAME = hostName + "-part-";
+        String fsUri = fileSystem.getUri().toString();
+        supportAppend = !fsUri.startsWith("s3") && !fsUri.startsWith("wasb"); // AWS EMR and Azure HDInsight

Review comment:
       may need to cover more cloud storage:
   - adls (azure data lake)
   - gs (google storage)
   - oss (aliyun oss) 
   
   or, can we just treat "hdfs" as the append case, all others as non-append?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org