You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "JonasJ-ap (via GitHub)" <gi...@apache.org> on 2023/01/26 02:37:34 UTC

[GitHub] [iceberg] JonasJ-ap commented on a diff in pull request #6571: Data: java api add GenericTaskWriter and add write demo to Doc.

JonasJ-ap commented on code in PR #6571:
URL: https://github.com/apache/iceberg/pull/6571#discussion_r1087385526


##########
docs/java-api.md:
##########
@@ -147,6 +147,58 @@ t.newAppend().appendFile(data).commit();
 t.commitTransaction();
 ```
 
+### WriteData
+
+The java api can write data into iceberg table.
+
+First write data to the data file, then submit the data file, the data you write will take effect in the table.
+
+For example, add 1000 pieces of data to the table.
+
+```java
+/**
+ * The structure of this table is the same as the demo table in java-api-quickstart
+ *
+ * Schema schema = new Schema(
+ *       Types.NestedField.required(1, "level", Types.StringType.get()),
+ *       Types.NestedField.required(2, "event_time", Types.TimestampType.withZone()),
+ *       Types.NestedField.required(3, "message", Types.StringType.get()),
+ *       Types.NestedField.optional(4, "call_stack", Types.ListType.ofRequired(5, Types.StringType.get()))
+ *   );
+ * PartitionSpec spec = PartitionSpec.builderFor(schema)
+ *       .hour("event_time")
+ *       .identity("level")
+ *       .build();
+ */
+
+GenericAppenderFactory appenderFactory = new GenericAppenderFactory(table.schema(), table.spec());
+int partitionId = 1, taskId = 1;
+FileFormat fileFormat = FileFormat.valueOf(table.properties().getOrDefault("write.format.default", "parquet").toUpperCase());
+OutputFileFactory outputFileFactory = OutputFileFactory.builderFor(table, partitionId, taskId).format(fileFormat).build();
+// TaskWriter write records into files. (the same is ok for unpartiton table)
+GenericTaskWriter<Record> genericTaskWriter = new GenericTaskWriter(table.spec(), fileFormat, appenderFactory, outputFileFactory, table.io(), 50L * 1024 * 1024);

Review Comment:
   Shall we make `50L * 1024 * 1024` a local variable named `targetFileSizeInBytes` so that readers can easily understand the meaning of this argument?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org