[jira] [Commented] (PHOENIX-1112) Atomically rebuild index partially when index update fails

["James Taylor (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PHOENIX-1112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14075434#comment-14075434 ] 

James Taylor commented on PHOENIX-1112:
---------------------------------------

Thanks for the patch, [~jeffreyz]. This is a big improvement. For easier review, it'd be good to spin up a pull request on our github mirror. [~jesse_yates] - would you please take a look, in particular at the change to how index write failures are handled. [~rajesh23] & [~jeffreyz] - what about local indexes? Should we *not* do this? Can we do the data & index writes in such a way that they're all-or-nothing (separate JIRA for this would be good)? 

Here's some feedback:
- I think we can get away with just one additional column in SYSTEM.CATALOG. I don't think we need INDEX_NEED_PARTIALLY_REBUILD. Just use a INDEX_DISABLE_TIMESTAMP value of 0 or null to know that a rebuild is not necessary.
- Did you run into any issues opening a Phoenix JDBC connection from the server-side in MetaDataRegionObserver? It would add a new dependency on the antlr jar being available on the server-side. Plus, is everything available from a coprocessor that we need (i.e. can it act just like an HBase client)?
- Is the change from calling recoveryWriter.writeAndKillYourselfOnFailure(indexUpdates) to unconditionally calling recoveryWriter.write(indexUpdates) intentional? Do we change our row-level (table row + index row) guarantees? [~jesse_yates] - this is the important bit for you to comment on. Should this be config parameter controlled? Maybe just a new no-op failure policy impl that could be configured by default even?
- Can you please use static constants for config parameter names (define them in QueryServices with the others) and static constants for default values (define them in QueryServicesOptions)?
- Would you mind filing a subtask to update the secondary index docs?

Good stuff, [~jeffreyz]!


> Atomically rebuild index partially when index update fails  
> ------------------------------------------------------------
>
>                 Key: PHOENIX-1112
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-1112
>             Project: Phoenix
>          Issue Type: Sub-task
>            Reporter: Jeffrey Zhong
>            Assignee: Jeffrey Zhong
>         Attachments: Phoenix-1112.patch
>
>
> This is a short-term work around & safe approach. Currently we disable an index when index update failed(still possible bring down the whole cluster). After an index is disable, human needs to be involved to rebuild entire index which maybe not ideal.
> The patch adds the support to automatically rebuild an disable index partially from where it failed. In addition, it removes RS abort during WAL recovery to prevent chain failures because we don't have to.
> To disable automatically rebuilding failed index, add the following configuration into hbase-site.xml:
> {noformat}
> <property>
>    <name>phoenix.index.failure.handling.rebuild</name>
>    <value>false</value>
> </property>
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)