[GitHub] [phoenix] gokceni commented on a change in pull request #913: PHOENIX-5435 - Annotate HBase WALs with Phoenix Metadata

[GitBox <gi...@apache.org>]

gokceni commented on a change in pull request #913:
URL: https://github.com/apache/phoenix/pull/913#discussion_r539564620



##########
File path: phoenix-core/src/it/java/org/apache/phoenix/end2end/index/ViewIndexIT.java
##########
@@ -72,6 +72,7 @@
 import org.apache.phoenix.util.PropertiesUtil;
 import org.apache.phoenix.util.QueryUtil;
 import org.apache.phoenix.util.SchemaUtil;
+import org.apache.phoenix.util.TestDDLUtil;

Review comment:
       ditto

##########
File path: phoenix-core/src/it/java/org/apache/phoenix/end2end/ImmutableTableIT.java
##########
@@ -17,6 +17,7 @@
  */
 package org.apache.phoenix.end2end;
 
+import org.apache.phoenix.util.TestDDLUtil;

Review comment:
       nit: is the import necessary? Seems like the only change on this file is this import

##########
File path: phoenix-core/src/main/java/org/apache/phoenix/coprocessor/GroupedAggregateRegionObserver.java
##########
@@ -400,7 +400,6 @@ protected boolean isRegionObserverFor(Scan scan) {
      * Used for an aggregate query in which the key order does not necessarily match the group by
      * key order. In this case, we must collect all distinct groups within a region into a map,
      * aggregating as we go.
-     * @param limit TODO

Review comment:
       TODO seems to be dropped but not implemented? 

##########
File path: phoenix-core/src/main/java/org/apache/phoenix/coprocessor/UngroupedAggregateRegionObserver.java
##########
@@ -73,6 +73,7 @@
 import org.apache.hadoop.io.WritableUtils;
 import org.apache.phoenix.coprocessor.generated.PTableProtos;
 import org.apache.phoenix.exception.SQLExceptionCode;
+import org.apache.phoenix.execute.MutationState;

Review comment:
       nit: lonely import

##########
File path: phoenix-core/src/main/java/org/apache/phoenix/coprocessor/UngroupedAggregateRegionScanner.java
##########
@@ -609,6 +613,9 @@ public boolean next(List<Cell> resultsToReturn) throws IOException {
                     } while (hasMore && (EnvironmentEdgeManager.currentTimeMillis() - startTime) < pageSizeInMs);
 
                     if (!mutations.isEmpty()) {
+                        if (isDelete || isUpsert) {
+                            annotateDataMutations(mutations, scan);
+                        }

Review comment:
       It would be nice to refactor this perhaps to a new method? Lines 596 to 602 is same as 616 to 622

##########
File path: phoenix-core/src/main/java/org/apache/phoenix/execute/MutationState.java
##########
@@ -682,6 +687,49 @@ private void generateMutations(final TableRef tableRef, final long mutationTimes
         values.putAll(modifiedValues);
     }
 
+    private void annotateMutationsWithMetadata(PTable table, List<Mutation> rowMutations) {
+        //only annotate if the change detection flag is on the table and HBase supports
+        // preWALAppend coprocs server-side
+        if (table == null || !table.isChangeDetectionEnabled()
+            || !HbaseCompatCapabilities.hasPreWALAppend()) {
+            return;
+        }
+        //annotate each mutation with enough metadata so that anyone interested can
+        // deterministically figure out exactly what Phoenix schema object created the mutation
+        // Server-side we can annotate the HBase WAL with these.
+        for (Mutation mutation : rowMutations) {
+            annotateMutationWithMetadata(table, mutation);
+        }
+
+    }
+
+    public static void annotateMutationWithMetadata(PTable table, Mutation mutation) {

Review comment:
       Actually why not use the this method in the Ungroupped..Scanner in that for loop that goes over mutations? Also mutation  durability was not checked there I think

##########
File path: phoenix-core/src/main/java/org/apache/phoenix/coprocessor/generated/ServerCachingProtos.java
##########
@@ -2192,6 +2192,21 @@ public Builder setColumnNameBytes(
      */
     com.google.protobuf.ByteString
         getParentTableTypeBytes();
+
+    // optional string logicalIndexName = 25;
+    /**
+     * <code>optional string logicalIndexName = 25;</code>
+     */
+    boolean hasLogicalIndexName();
+    /**
+     * <code>optional string logicalIndexName = 25;</code>
+     */
+    java.lang.String getLogicalIndexName();

Review comment:
       Would this be populated for both view indexes and regular indexes and local indexes?

##########
File path: phoenix-core/src/main/java/org/apache/phoenix/execute/MutationState.java
##########
@@ -674,6 +678,7 @@ private void generateMutations(final TableRef tableRef, final long mutationTimes
                         mutation.setAttribute(PhoenixIndexBuilder.ATOMIC_OP_ATTRIB, onDupKeyBytes);
                     }
                 }
+                annotateMutationsWithMetadata(table, rowMutations);

Review comment:
       perhaps move this to line 684? (out of if else?)




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org