[jira] [Comment Edited] (PHOENIX-5156) Consistent Global Indexes for Non-Transactional Tables

["Kadir OZDEMIR (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PHOENIX-5156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16779014#comment-16779014 ] 

Kadir OZDEMIR edited comment on PHOENIX-5156 at 2/27/19 8:04 AM:
-----------------------------------------------------------------

[~lhofhansl], it is not fair to kill an idea because the proposed implementation of it has been chosen to be on the server side. Also, it is not fair to claim that implementing something on the server side is a bad idea because we have seen some issues with our server side implementations. As far as I know, based on my discussions with other Phoenix developers, we do not have a good handle on the root causes of our issues. I was initially going with the general belief that server-server RPC communications are the main root cause of our index inconsistency issues until two of our developers warned me that we were not sure about it. This proposal does not address the root cause of these issues but correctly deals with them and does not allow indexes to return wrong data.

I thank you for informing me that some of our scans happen on the client side. I think I have read about it but completely forgot about it. For scans on the client side, we need to check the VERIFIED column on the client side too and read from the data table when needed. Doing this on the client side is much easier than doing it server side as we do not need to serialize and send some metadata to the server side in this case. I do not see any problem with that. I will look into QueryOptimize code and add details on how to do this.  


was (Author: kozdemir):
[~lhofhansl], it is not fair to kill an idea because the proposed implementation of it has been chosen to be on the server side. Also, it is not fair to claim that implementing something on the server side is a bad idea because we have seen some issues with our server side implementations. As far as I know, based on my discussions with other Phoenix developers, we do not have a good handle on the root causes of our issues. I was initially going with the general belief that server-server RPC communications are the main root cause of our index inconsistency issues until two of our developers warned me that we were not sure about it. This proposal does not address the root cause of these issues but correctly deals with them and does not allow indexes to return wrong data.

I thank you for informing me that some of our scans happen on the client side. I think I have read about it but completely forgot about it. For scans on the client side, we need to check the VERIFIED column on the client side too and read from the data table when needed. Doing this on the client side is much easier than doing it server side as we do not need to serialize and send some metadata server side in this case. I do not see any problem with that. I will look into QueryOptimize code and add details on how to do this.  

> Consistent Global Indexes for Non-Transactional Tables
> ------------------------------------------------------
>
>                 Key: PHOENIX-5156
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-5156
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 4.13.0, 4.14.0, 5.0.0, 4.14.1
>            Reporter: Kadir OZDEMIR
>            Assignee: Kadir OZDEMIR
>            Priority: Major
>
> Without transactional tables, the global indexes can get easily out of sync with their data tables in Phoenix. Transactional tables require a separate transaction manager, have some restrictions and performance penalties. This issue is to have consistent global indexes without the need for using transactional tables.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)