[jira] [Commented] (PHOENIX-4855) Continue to write base table column metadata when creating a view in order to support rollback

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PHOENIX-4855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16598106#comment-16598106 ] 

ASF GitHub Bot commented on PHOENIX-4855:
-----------------------------------------

GitHub user twdsilva opened a pull request:

    https://github.com/apache/phoenix/pull/339

    PHOENIX-4855 Continue to write base table column metadata when creati…

    …ng a view in order to support rollback
    
    Previously in MetadataEndpointImpl we were removing parent column metadata while creating a child view. We still need to write the parent columns in order to be able to rollback if needed. This PR filters out the parent columns while creating the PTable of a child view. When a view is resolved we also resolve all its parents and add their columns (see MetadataEndpointImpl.addDerivedColumnsFromAncestors). 
    I added some more tests for salted tables. I also changed PhoenixDatabaseMetaData.getPrimaryKeys() to load the PTable and return the metadata instead of directly querying the SYSTEM.CATALOG table (which will not work any more since we don't store columns inherited from parents in a view)
    
    @ankitsinghal  @ChinmaySKulkarni  can you please review?

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/twdsilva/phoenix PHOENIX-4855

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/phoenix/pull/339.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #339
    
----
commit a352c6ec72204cda7c09556e296139b7d5cc92dd
Author: Thomas D'Silva <td...@...>
Date:   2018-08-20T17:42:56Z

    PHOENIX-4855 Continue to write base table column metadata when creating a view in order to support rollback

----


> Continue to write base table column metadata when creating a view in order to support rollback
> ----------------------------------------------------------------------------------------------
>
>                 Key: PHOENIX-4855
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-4855
>             Project: Phoenix
>          Issue Type: Sub-task
>            Reporter: Thomas D'Silva
>            Assignee: Thomas D'Silva
>            Priority: Major
>
> Currently we filter out the parent table columns while creating a view. If we have to support rollback we still need to write the parent table column metadata. 
> Instead of this we should filter out the duplicate columns in {{MetaDataEndpointImpl.combineColumns}}
> {code}
> // Here we are passed the parent's columns to add to a view, PHOENIX-3534 allows for a splittable
>             // System.Catalog thus we only store the columns that are new to the view, not the parents columns,
>             // thus here we remove everything that is ORDINAL.POSITION <= baseColumnCount and update the
>             // ORDINAL.POSITIONS to be shifted accordingly.
>             // TODO PHOENIX-4767 remove the following code that removes the base table column metadata in the next release 
>             if (PTableType.VIEW.equals(tableType) && !ViewType.MAPPED.equals(viewType)) {
>                 boolean isSalted = MetaDataUtil.getSaltBuckets(tableMetadata, GenericKeyValueBuilder.INSTANCE, new ImmutableBytesWritable()) > 0;
>                 int baseColumnCount = MetaDataUtil.getBaseColumnCount(tableMetadata) - (isSalted ? 1 : 0);
>                 if (baseColumnCount > 0) {
>                     Iterator<Mutation> mutationIterator = tableMetadata.iterator();
>                     while (mutationIterator.hasNext()) {
>                         Mutation mutation = mutationIterator.next();
>                         // if not null and ordinal position < base column count remove this mutation
>                         ImmutableBytesWritable ptr = new ImmutableBytesWritable();
>                         MetaDataUtil.getMutationValue(mutation, PhoenixDatabaseMetaData.ORDINAL_POSITION_BYTES,
>                             GenericKeyValueBuilder.INSTANCE, ptr);
>                         if (MetaDataUtil.getMutationValue(mutation, PhoenixDatabaseMetaData.ORDINAL_POSITION_BYTES,
>                             GenericKeyValueBuilder.INSTANCE, ptr)) {
>                             int ordinalValue = PInteger.INSTANCE.getCodec().decodeInt(ptr, SortOrder.ASC);
>                             if (ordinalValue <= baseColumnCount) {
>                                 mutationIterator.remove();
>                             } else {
>                                 if (mutation instanceof Put) {
>                                     byte[] ordinalPositionBytes = new byte[PInteger.INSTANCE.getByteSize()];
>                                     int newOrdinalValue = ordinalValue - baseColumnCount;
>                                     PInteger.INSTANCE.getCodec()
>                                         .encodeInt(newOrdinalValue, ordinalPositionBytes, 0);
>                                     byte[] family = Iterables.getOnlyElement(mutation.getFamilyCellMap().keySet());
>                                     MetaDataUtil.mutatePutValue((Put) mutation, family, PhoenixDatabaseMetaData.ORDINAL_POSITION_BYTES, ordinalPositionBytes);
>                                 }
>                             }
>                         }
>                     }
>                 }
>             }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)