[jira] [Commented] (PHOENIX-5373) GlobalIndexChecker should treat the rows created by the previous design as unverified

["Hadoop QA (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PHOENIX-5373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16873774#comment-16873774 ] 

Hadoop QA commented on PHOENIX-5373:
------------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12973008/PHOENIX-5373.master.001.patch
  against master branch at commit c4d75ddeb5134e245dbe9019481758f81fb30e9d.
  ATTACHMENT ID: 12973008

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:red}-1 tests included{color}.  The patch doesn't appear to include any new or modified tests.
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of javac compiler warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number of release audit warnings.

    {color:red}-1 lineLengths{color}.  The patch introduces the following lines longer than 100:
    +        clientProps.put(QueryServices.INDEX_REGION_OBSERVER_ENABLED_ATTRIB, Boolean.FALSE.toString());
+        setUpTestDriver(ReadOnlyProps.EMPTY_PROPS, new ReadOnlyProps(clientProps.entrySet().iterator()));
+            // Due to PHOENIX-5375 and PHOENIX-5376, the useSnapshot and bulk load options are ignored for global indexes
+                            // Due to PHOENIX-5376, the bulk load option is ignored for global indexes
+            // Writing index directly will generate unverified rows with no corresponding data rows. These rows will not be visible to the applications
+                assertEquals("Expected data table row count to match expected:<2> but was:<1>", e.getMessage());
+                dataHTable = ServerUtil.ConnectionFactory.getConnection(ServerUtil.ConnectionType.DEFAULT_SERVER_CONNECTION, env).getTable(TableName.valueOf(dataTableName));
+            LOG.debug("repairIndexRows : " + Bytes.toStringBinary(dataRowKey) + " --> " + Bytes.toStringBinary(indexRowKey));

     {color:red}-1 core tests{color}.  The patch failed these unit tests:
     ./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.SaltedIndexIT

Test results: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2707//testReport/
Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2707//console

This message is automatically generated.

> GlobalIndexChecker should treat the rows created by the previous design as unverified 
> --------------------------------------------------------------------------------------
>
>                 Key: PHOENIX-5373
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-5373
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 5.0.0, 4.14.2
>            Reporter: Kadir OZDEMIR
>            Assignee: Kadir OZDEMIR
>            Priority: Major
>         Attachments: PHOENIX-5373.master.001.patch
>
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> For the ease of transition from the old global secondary index design to the new one (without having read performance impact), GlobalIndexChecker treats existing index rows (i.e., the rows created by the previous design) as verified. We have discovered that this would lead to keeping stale index rows around forever and including them in the result of queries. A stale index row is a row for which we do not have the corresponding data table row. The reason that we do not have the data table row is either the row is deleted (but not the corresponding index row(s)), or the data table and index rows are written with different timestamps. The assumption was that such rows would be fixed by index rebuild. Unfortunately, without dropping or truncating index tables, these stale rows may not be fixed by index rebuild. Thus, GlobalIndexChecker should treat the rows created by the previous design as unverified.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)