[GitHub] [gobblin] ZihanLi58 commented on a diff in pull request #3672: [GOBBLIN-1810] Support general iceberg catalog in icebergMetadataWriter

["ZihanLi58 (via GitHub)" <gi...@apache.org>]

ZihanLi58 commented on code in PR #3672:
URL: https://github.com/apache/gobblin/pull/3672#discussion_r1164651755


##########
gobblin-iceberg/src/main/java/org/apache/gobblin/iceberg/writer/IcebergMetadataWriter.java:
##########
@@ -801,12 +813,14 @@ public void flush(String dbName, String tableName) throws IOException {
         Transaction transaction = tableMetadata.transaction.get();
         Map<String, String> props = tableMetadata.newProperties.or(
             Maps.newHashMap(tableMetadata.lastProperties.or(getIcebergTable(tid).properties())));
-        String topic = props.get(TOPIC_NAME_KEY);
+        //Set data offset range
+        setDatasetOffsetRange(tableMetadata, props);
+        String topicName = getTopicName(tid, tableMetadata);

Review Comment:
   We are still fallback to the previous topic name calculated from NativeName here in case datasetOffsetRange does not exist.  Also GTE require this datasetName to set the HDFS dataset path information



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org