[GitHub] rdblue commented on a change in pull request #3: Store multiple partition specs in table metadata.

[GitBox <gi...@apache.org>]

rdblue commented on a change in pull request #3: Store multiple partition specs in table metadata.
URL: https://github.com/apache/incubator-iceberg/pull/3#discussion_r236341633
 
 

 ##########
 File path: core/src/main/java/com/netflix/iceberg/TableMetadata.java
 ##########
 @@ -239,15 +255,42 @@ public Snapshot currentSnapshot() {
 
   public TableMetadata updateTableLocation(String newLocation) {
     return new TableMetadata(ops, null, newLocation,
-        System.currentTimeMillis(), lastColumnId, schema, spec, properties, currentSnapshotId,
-        snapshots, snapshotLog);
+        System.currentTimeMillis(), lastColumnId, schema, defaultSpecId, specs, properties,
+        currentSnapshotId, snapshots, snapshotLog);
   }
 
   public TableMetadata updateSchema(Schema schema, int lastColumnId) {
+    PartitionSpec.checkCompatibility(spec(), schema);
+    return new TableMetadata(ops, null, location,
+        System.currentTimeMillis(), lastColumnId, schema, defaultSpecId, specs, properties,
+        currentSnapshotId, snapshots, snapshotLog);
+  }
+
+  public TableMetadata updatePartitionSpec(PartitionSpec spec) {
     PartitionSpec.checkCompatibility(spec, schema);
+
+    // if the spec already exists, use the same ID. otherwise, use 1 more than the highest ID.
+    int newDefaultSpecId = 0;
+    for (Map.Entry<Integer, PartitionSpec> entry : specs.entrySet()) {
+      if (spec.equals(entry.getValue())) {
+        newDefaultSpecId = entry.getKey();
+        break;
+      } else if (newDefaultSpecId <= entry.getKey()) {
+        newDefaultSpecId = entry.getKey() + 1;
+      }
+    }
+
+    Preconditions.checkArgument(defaultSpecId != newDefaultSpecId,
 
 Review comment:
   This validates that the spec has changed. It is assumed that if updatePartitionSpec is called, the intent was to change the spec. In buildReplacement, the entire table state will be replaced. The new table can have the same partition spec as the old, or it can use the same one. There isn't a requirement to change the spec.
   
   There may be a situation where a user attempts to update the partition spec without actually changing it, but I thought that this should be conservative and validate. If the spec doesn't actually change because a user attempted to update the spec to the current one, then some other component should catch it and avoid committing a change entirely.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services