[jira] [Commented] (PHOENIX-6373) Schema changes that require table re-writes can be supported (Online data format changes)

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PHOENIX-6373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17468849#comment-17468849 ] 

ASF GitHub Bot commented on PHOENIX-6373:
-----------------------------------------

gjacoby126 commented on a change in pull request #1368:
URL: https://github.com/apache/phoenix/pull/1368#discussion_r778370600



##########
File path: phoenix-core/src/it/java/org/apache/phoenix/end2end/TransformToolIT.java
##########
@@ -406,6 +408,71 @@ public void testTransformIndex() throws Exception {
         }
     }
 
+    @Test
+    public void testTransformMutationFailureRepair() throws Exception {
+        String schemaName = generateUniqueName();
+        String dataTableName = generateUniqueName();
+        String dataTableFullName = SchemaUtil.getTableName(schemaName, dataTableName);
+
+        Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
+        try (Connection conn = DriverManager.getConnection(getUrl(), props)) {
+            conn.setAutoCommit(true);
+            IndexRegionObserver.setFailPreIndexUpdatesForTesting(true);
+            int numOfRows = 0;
+            createTableAndUpsertRows(conn, dataTableFullName, numOfRows);
+            SingleCellIndexIT.assertMetadata(conn, PTable.ImmutableStorageScheme.ONE_CELL_PER_COLUMN, PTable.QualifierEncodingScheme.NON_ENCODED_QUALIFIERS, dataTableFullName);
+
+            conn.createStatement().execute("ALTER TABLE " + dataTableFullName +
+                    " SET IMMUTABLE_STORAGE_SCHEME=SINGLE_CELL_ARRAY_WITH_OFFSETS, COLUMN_ENCODED_BYTES=2");
+            SystemTransformRecord record = Transform.getTransformRecord(schemaName, dataTableName, null, null, conn.unwrap(PhoenixConnection.class));
+            assertNotNull(record);
+            assertMetadata(conn, PTable.ImmutableStorageScheme.SINGLE_CELL_ARRAY_WITH_OFFSETS, PTable.QualifierEncodingScheme.TWO_BYTE_QUALIFIERS, record.getNewPhysicalTableName());
+
+            String upsertQuery = String.format("UPSERT INTO %s VALUES(?, ?, ?)", dataTableFullName);
+            PreparedStatement stmt1 = conn.prepareStatement(upsertQuery);
+            try {
+                IndexToolIT.upsertRow(stmt1, 1);
+                fail("Transform table upsert should have failed");
+            } catch (Exception e) {
+            }
+            assertEquals(0, getRowCount(conn, record.getNewPhysicalTableName()));
+            assertEquals(0, getRowCount(conn,dataTableFullName));
+
+            IndexRegionObserver.setFailPreIndexUpdatesForTesting(false);
+            IndexRegionObserver.setFailPostIndexUpdatesForTesting(true);
+            IndexToolIT.upsertRow(stmt1, 2);
+
+            assertEquals(1, getRowCount(conn, record.getNewPhysicalTableName()));
+            assertEquals(1, getRowCount(conn,dataTableFullName));
+
+            IndexRegionObserver.setFailPostIndexUpdatesForTesting(false);
+            IndexRegionObserver.setFailDataTableUpdatesForTesting(true);
+            try {
+                IndexToolIT.upsertRow(stmt1, 3);
+                fail("Data table upsert should have failed");
+            } catch (Exception e) {
+            }
+            assertEquals(2, getRowCount(conn, record.getNewPhysicalTableName()));
+            assertEquals(1, getRowCount(conn,dataTableFullName));
+
+            IndexRegionObserver.setFailDataTableUpdatesForTesting(false);
+
+            List<String> args = getArgList(schemaName, dataTableName, null,
+                    null, null, null, false, false, false, false);
+            runTransformTool(args.toArray(new String[0]), 0);
+
+            //TODO: Implement GlobalIndexChecker like repair for unverified rows (W-9783508)

Review comment:
       Change TODO to remove work item

##########
File path: phoenix-core/src/it/java/org/apache/phoenix/end2end/index/TransformIT.java
##########
@@ -36,10 +40,8 @@
 import static org.apache.phoenix.schema.PTable.ImmutableStorageScheme.ONE_CELL_PER_COLUMN;
 import static org.apache.phoenix.schema.PTable.QualifierEncodingScheme.NON_ENCODED_QUALIFIERS;
 import static org.apache.phoenix.util.TestUtil.TEST_PROPERTIES;
-import static org.junit.Assert.assertEquals;
-import static org.junit.Assert.assertFalse;
-import static org.junit.Assert.assertTrue;
-import static org.junit.Assert.fail;
+import static org.apache.phoenix.util.TestUtil.getRowCount;
+import static org.junit.Assert.*;

Review comment:
       nit: no star imports




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> Schema changes that require table re-writes can be supported (Online data format changes)
> -----------------------------------------------------------------------------------------
>
>                 Key: PHOENIX-6373
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-6373
>             Project: Phoenix
>          Issue Type: Improvement
>            Reporter: Gokcen Iskender
>            Assignee: Gokcen Iskender
>            Priority: Major
>
> Today, using ALTER TABLE or ALTER INDEX commands, the user can make certain changes to the schema. For example, changing certain table properties like TTL and immutability, adding nullable columns/dropping non-pk columns are allowed as well as certain index state changes. All of the allowed changes don’t require the table to be re-written. As soon as the ALTER command returns, most of the changes became available immediately (eg. index disable, TTL) but some of them might take some time and the syntax lets you to specify async (eg. Rebuild index) and depending on the client cache settings, some changes never make it to the client (eg. select * that run from a client never seeing the new column since its schema cache is not updated). 
> If the user wants to change the schema properties that require table re-writes, it is blocked and ALTER fails. Phoenix lacks of the ability to change some of the table schemas and attributes, such changing the row key (primary keys), the type of a column, the table storage format, the column encoding, etc. There is no way to make this changes with no or very minimal service interruption.
> Design doc link: [https://docs.google.com/document/d/1D24zRETMEetXvc3MSZj9WKYeUnooQX5BLv6gSDEmwfk/edit?usp=sharing]



--
This message was sent by Atlassian Jira
(v8.20.1#820001)