Index Maintenance on Secondary Indices Marked as UNUSABLE

[Julian Jaffe <jj...@marinsoftware.com>]

Hi all,

Does Phoenix use the same execution path for updating indices mark as
UNUSABLE as it does for USABLE indices (that is, will indices marked as
UNUSABLE require an additional write for every upsert, just as USABLE
indices will)? If maintaining an unsuable index still requires an extra
write, is there a way instead to disable an index without forcing a full
rebuild if it's reenabled? We're replacing a few indices and are trying to
ensure we can roll back quickly if something goes wrong instead of having
to wait to rebuild a large index.

Thanks,
Julian Jaffe

Re: Index Maintenance on Secondary Indices Marked as UNUSABLE

[James Taylor <ja...@apache.org>]

Actually, the partial rebuild would need to work differently than I
mentioned to handle the case where rows are deleted from the table while
the index is disabled. See my comment here[1] for a pointer to some code
that handles this correctly.

[1]
https://issues.apache.org/jira/browse/PHOENIX-2890?focusedCommentId=15280249&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15280249

On Mon, Aug 15, 2016 at 5:12 PM, James Taylor <ja...@apache.org>
wrote:

> Hi Julian,
> Yes, marking an index as UNUSABLE simply means it won't be considered at
> query time. It will still be maintained. If you DISABLE it, then you won't
> be hit with the maintenance cost, but we don't have a mechanism to
> partially rebuild it. See PHOENIX-2890 and the WIP patch. In theory, if you
> get the cell time stamp of the INDEX_STATE cell (i.e. when it was set to
> DISABLE), don't attempt to delete existing rows for the index, and then use
> this time stamp as the min time stamp of the UPSERT SELECT that rebuilds
> the index, it should just work (sens edge cases of any out of order data
> row updates).
> Thanks,
> James
>
>
>
> On Mon, Aug 15, 2016 at 4:46 PM, Julian Jaffe <jj...@marinsoftware.com>
> wrote:
>
>> Hi all,
>>
>> Does Phoenix use the same execution path for updating indices mark as
>> UNUSABLE as it does for USABLE indices (that is, will indices marked as
>> UNUSABLE require an additional write for every upsert, just as USABLE
>> indices will)? If maintaining an unsuable index still requires an extra
>> write, is there a way instead to disable an index without forcing a full
>> rebuild if it's reenabled? We're replacing a few indices and are trying to
>> ensure we can roll back quickly if something goes wrong instead of having
>> to wait to rebuild a large index.
>>
>> Thanks,
>> Julian Jaffe
>>
>
>

Re: Index Maintenance on Secondary Indices Marked as UNUSABLE

[James Taylor <ja...@apache.org>]

Hi Julian,
Yes, marking an index as UNUSABLE simply means it won't be considered at
query time. It will still be maintained. If you DISABLE it, then you won't
be hit with the maintenance cost, but we don't have a mechanism to
partially rebuild it. See PHOENIX-2890 and the WIP patch. In theory, if you
get the cell time stamp of the INDEX_STATE cell (i.e. when it was set to
DISABLE), don't attempt to delete existing rows for the index, and then use
this time stamp as the min time stamp of the UPSERT SELECT that rebuilds
the index, it should just work (sens edge cases of any out of order data
row updates).
Thanks,
James



On Mon, Aug 15, 2016 at 4:46 PM, Julian Jaffe <jj...@marinsoftware.com>
wrote:

> Hi all,
>
> Does Phoenix use the same execution path for updating indices mark as
> UNUSABLE as it does for USABLE indices (that is, will indices marked as
> UNUSABLE require an additional write for every upsert, just as USABLE
> indices will)? If maintaining an unsuable index still requires an extra
> write, is there a way instead to disable an index without forcing a full
> rebuild if it's reenabled? We're replacing a few indices and are trying to
> ensure we can roll back quickly if something goes wrong instead of having
> to wait to rebuild a large index.
>
> Thanks,
> Julian Jaffe
>