[jira] [Commented] (PHOENIX-3534) Support multi region SYSTEM.CATALOG table

["Hudson (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PHOENIX-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16625993#comment-16625993 ] 

Hudson commented on PHOENIX-3534:
---------------------------------

FAILURE: Integrated in Jenkins build Phoenix-omid2 #89 (See [https://builds.apache.org/job/Phoenix-omid2/89/])
PHOENIX-3534 Support multi region SYSTEM.CATALOG table (Thomas D'Silva (tdsilva: rev 93fdd5bad22cde313c9f34fe7448dca44377a27c)
* (add) phoenix-core/src/it/java/org/apache/phoenix/end2end/SplitSystemCatalogTests.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/compile/UnionCompiler.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/query/QueryConstants.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/compile/CreateTableCompiler.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/schema/PTable.java
* (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/AlterTableIT.java
* (edit) phoenix-core/src/test/java/org/apache/phoenix/query/BaseTest.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
* (edit) phoenix-core/src/test/java/org/apache/phoenix/util/MetaDataUtilTest.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/compile/UpsertCompiler.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixConnection.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java
* (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/TenantSpecificViewIndexIT.java
* (edit) phoenix-core/src/it/java/org/apache/phoenix/execute/PartialCommitIT.java
* (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/ChildViewsUseParentViewIndexIT.java
* (edit) pom.xml
* (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/MutableIndexIT.java
* (edit) phoenix-core/src/it/java/org/apache/phoenix/replication/SystemCatalogWALEntryFilterIT.java
* (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/SystemCatalogCreationOnConnectionIT.java
* (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/BaseIndexIT.java
* (edit) phoenix-core/src/test/java/org/apache/phoenix/util/TestUtil.java
* (edit) phoenix-protocol/src/main/PTable.proto
* (edit) phoenix-core/src/main/java/org/apache/phoenix/schema/TableProperty.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServices.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/parse/ParseNodeFactory.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionlessQueryServicesImpl.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/schema/PMetaDataImpl.java
* (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/SystemCatalogIT.java
* (edit) phoenix-core/src/it/java/org/apache/phoenix/rpc/UpdateCacheIT.java
* (edit) phoenix-core/src/it/java/org/apache/phoenix/coprocessor/StatisticsCollectionRunTrackerIT.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/compile/FromCompiler.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/compile/TraceQueryPlan.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/schema/PTableImpl.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataSplitPolicy.java
* (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/IndexMetadataIT.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/util/MetaDataUtil.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java
* (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/PhoenixDriverIT.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/parse/DropTableStatement.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/coprocessor/generated/PTableProtos.java
* (add) phoenix-core/src/main/java/org/apache/phoenix/coprocessor/TableInfo.java
* (add) phoenix-core/src/main/java/org/apache/phoenix/coprocessor/ViewFinder.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/compile/ListJarsQueryPlan.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixStatement.java
* (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/ExplainPlanWithStatsEnabledIT.java
* (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/ViewIT.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/coprocessor/generated/MetaDataProtos.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java
* (edit) phoenix-core/src/test/java/org/apache/phoenix/compile/QueryCompilerTest.java
* (add) phoenix-core/src/main/java/org/apache/phoenix/coprocessor/WhereConstantParser.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/util/UpgradeUtil.java
* (add) phoenix-core/src/it/java/org/apache/phoenix/end2end/StatsEnabledSplitSystemCatalogIT.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/util/PhoenixRuntime.java
* (edit) phoenix-protocol/src/main/MetaDataService.proto
* (add) phoenix-core/src/it/java/org/apache/phoenix/end2end/SplitSystemCatalogIT.java
* (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/DropColumnIT.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixDatabaseMetaData.java
* (edit) phoenix-core/src/test/java/org/apache/phoenix/execute/LiteralResultIteratorPlanTest.java
* (edit) phoenix-core/src/test/java/org/apache/phoenix/execute/UnnestArrayPlanTest.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataProtocol.java
* (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/UpgradeIT.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/query/QueryServices.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/schema/PColumn.java
* (add) phoenix-core/src/main/java/org/apache/phoenix/coprocessor/TableViewFinderResult.java
* (edit) phoenix-core/src/test/java/org/apache/phoenix/expression/ColumnExpressionTest.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/util/IndexUtil.java
* (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/AppendOnlySchemaIT.java
* (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/BaseTenantSpecificViewIndexIT.java
* (add) phoenix-core/src/test/java/org/apache/phoenix/coprocessor/MetaDataEndpointImplTest.java
* (add) phoenix-core/src/main/java/org/apache/phoenix/schema/ParentTableNotFoundException.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/schema/DelegateColumn.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/expression/LikeExpression.java
* (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/AlterMultiTenantTableWithViewsIT.java
* (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/TenantSpecificTablesDDLIT.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/schema/PTableKey.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/query/DelegateConnectionQueryServices.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/util/SchemaUtil.java
* (add) phoenix-core/src/main/java/org/apache/phoenix/compile/ColumnNameTrackingExpressionCompiler.java
* (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/AlterTableWithViewsIT.java
* (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/MigrateSystemTablesToSystemNamespaceIT.java
* (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/QueryDatabaseMetaDataIT.java
* (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/ViewIndexIT.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/replication/SystemCatalogWALEntryFilter.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/schema/PColumnImpl.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/schema/SaltingUtil.java
* (edit) phoenix-core/src/test/java/org/apache/phoenix/execute/CorrelatePlanTest.java
* (delete) phoenix-core/src/it/java/org/apache/phoenix/end2end/SaltedViewIT.java


> Support multi region SYSTEM.CATALOG table
> -----------------------------------------
>
>                 Key: PHOENIX-3534
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-3534
>             Project: Phoenix
>          Issue Type: Bug
>            Reporter: James Taylor
>            Assignee: Thomas D'Silva
>            Priority: Major
>             Fix For: 4.15.0, 5.1.0
>
>         Attachments: PHOENIX-3534-v2.patch, PHOENIX-3534-v3.patch, PHOENIX-3534.patch
>
>
> Currently Phoenix requires that the SYSTEM.CATALOG table is single region based on the server-side row locks being held for operations that impact a table and all of it's views. For example, adding/removing a column from a base table pushes this change to all views.
> As an alternative to making the SYSTEM.CATALOG transactional (PHOENIX-2431), when a new table is created we can do a lazy cleanup  of any rows that may be left over from a failed DDL call (kudos to [~lhofhansl] for coming up with this idea). To implement this efficiently, we'd need to also do PHOENIX-2051 so that we can efficiently find derived views.
> The implementation would rely on an optimistic concurrency model based on checking our sequence numbers for each table/view before/after updating. Each table/view row would be individually locked for their change (metadata for a view or table cannot span regions due to our split policy), with the sequence number being incremented under lock and then returned to the client.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)