You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2020/12/04 17:11:08 UTC
[GitHub] [iceberg] RussellSpitzer commented on a change in pull request #1525: Provide API and Implementation for Creating Iceberg Tables from Spark
RussellSpitzer commented on a change in pull request #1525:
URL: https://github.com/apache/iceberg/pull/1525#discussion_r536244613
##########
File path: spark/src/main/java/org/apache/iceberg/actions/Actions.java
##########
@@ -77,6 +82,81 @@ public ExpireSnapshotsAction expireSnapshots() {
return new ExpireSnapshotsAction(spark, table);
}
+ /**
+ * Converts the provided table into an Iceberg table in place. The table will no longer be represented by it's
+ * previous provider in the session catalog and a new metadata directory will be created at the table's location.
Review comment:
woops only converted the lower one, this one is fixed too now
##########
File path: spark/src/main/java/org/apache/iceberg/actions/Actions.java
##########
@@ -77,6 +82,120 @@ public ExpireSnapshotsAction expireSnapshots() {
return new ExpireSnapshotsAction(spark, table);
}
+ /**
+ * Converts the provided table into an Iceberg table in place. The table will no longer be represented by it's
+ * previous provider in the session catalog and a new metadata directory will be created at the table's location.
+ *
+ * @param tableName Table to be converted
+ * @return {@link CreateAction} to perform migration
+ */
+ public static CreateAction migrate(String tableName) {
+ try {
+ return DynMethods.builder("migrate")
+ .impl(implClass(), String.class).buildStaticChecked()
+ .invoke(tableName);
+ } catch (NoSuchMethodException ex) {
+ throw new UnsupportedOperationException("Migrate is not implemented for this version of Spark");
+ }
+ }
+
+ /**
+ * Converts the provided table into an Iceberg table in place. The table will no longer be accesible by it's
+ * previous implementation
+ *
+ * @param tableName Table to be converted
+ * @param spark Spark session to use for looking up table
+ * @return {@link CreateAction} to perform migration
+ */
+ public static CreateAction migrate(SparkSession spark, String tableName) {
+ try {
+ return DynMethods.builder("migrate")
+ .impl(implClass(), SparkSession.class, String.class).buildStaticChecked()
+ .invoke(spark, tableName);
+ } catch (NoSuchMethodException ex) {
+ throw new UnsupportedOperationException("Migrate is not implemented for this version of Spark");
+ }
+ }
+
+ /**
+ * Creates an independent Iceberg table based on a given table. The new Iceberg table can be altered, appended or
+ * deleted without causing any change to the original. New data and metadata will be created in the default
+ * location for tables of this name in the destination catalog.
+ *
+ * @param sourceTable Original table which is the basis for the new Iceberg table
+ * @param destTable New Iceberg table being created
+ * @return {@link CreateAction} to perform snapshot
+ */
+ public static CreateAction snapshot(SparkSession spark, String sourceTable, String destTable) {
+ try {
+ return DynMethods.builder("snapshot")
+ .impl(implClass(), SparkSession.class, String.class, String.class).buildStaticChecked()
+ .invoke(spark, sourceTable, destTable);
+ } catch (NoSuchMethodException ex) {
+ throw new UnsupportedOperationException("Snapshot is not implemented for this version of Spark");
+ }
+ }
+
+ /**
+ * Creates an independent Iceberg table based on a given table. The new Iceberg table can be altered, appended or
+ * deleted without causing any change to the original. New data and metadata will be created in the default
+ * location for tables of this name in the destination catalog.
+ *
+ * @param sourceTable Original table which is the basis for the new Iceberg table
+ * @param destTable New Iceberg table being created
+ * @return {@link CreateAction} to perform snapshot
+ */
+ public static CreateAction snapshot(String sourceTable, String destTable) {
+ try {
+ return DynMethods.builder("snapshot")
+ .impl(implClass(), String.class, String.class).buildStaticChecked()
+ .invoke(sourceTable, destTable);
+ } catch (NoSuchMethodException ex) {
+ throw new UnsupportedOperationException("Snapshot is not implemented for this version of Spark");
+ }
+ }
+
+ /**
+ * Creates an independent Iceberg table based on a given table. The new Iceberg table can be altered, appended or
+ * deleted without causing any change to the original. New data and metadata will be created in the
+ * new location passed to this method.
+ *
+ * @param sourceTable Original table which is the basis for the new Iceberg table
+ * @param destTable New Iceberg table being created
+ * @param location Location for metadata and new data for the new table
+ * @return {@link CreateAction} to perform snapshot
+ */
+ public static CreateAction snapshot(String sourceTable, String destTable, String location) {
Review comment:
Removed, added another Interface SnapshotAction, with "withLocation" method for supplying location
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org