You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@phoenix.apache.org by "James Taylor (JIRA)" <ji...@apache.org> on 2014/04/24 06:05:14 UTC
[jira] [Comment Edited] (PHOENIX-935) create local index table with
the same split keys of user table
[ https://issues.apache.org/jira/browse/PHOENIX-935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13979254#comment-13979254 ]
James Taylor edited comment on PHOENIX-935 at 4/24/14 4:04 AM:
---------------------------------------------------------------
Thanks, [~rajesh23]. Here's some feedback:
You won't need the following change, as this is a facility to pre-split a table. We won't allow a local index to be pre-split or salted -this is an error case we catch earlier:
{code}
- splits = SchemaUtil.processSplits(splits, pkColumns, saltBucketNum, connection.getQueryServices().getProps().getBoolean(
+ if (parent != null && tableType == PTableType.INDEX && indexType == IndexType.LOCAL) {
+ tableProps.put(MetaDataUtil.PARENT_TABLE_KEY, parent.getPhysicalName().getString());
+ tableProps.put(MetaDataUtil.IS_LOCAL_INDEX_TABLE_PROP_NAME, Boolean.TRUE);
+ splits = getSplitKeys(connection.getQueryServices().getAllTableRegions(parent.getPhysicalName().getBytes()));
+ } else {
+ splits = SchemaUtil.processSplits(splits, pkColumns, saltBucketNum, connection.getQueryServices().getProps().getBoolean(
QueryServices.ROW_KEY_ORDER_SALTED_TABLE_ATTRIB, QueryServicesOptions.DEFAULT_ROW_KEY_ORDER_SALTED_TABLE));
+ }
{code}
The new function you added can be simplified, as we don't need to calculate the MAX_FILE_SIZE as a percentage of the data table MAX_FILE_SIZE. We do this for global indexes, as they're typically smaller and you'd want to split them when they're smaller to attempt to have the same number of index regions as data regions (you get better parallelization in that case). For local indexes, we don't need to do this.
{code}
+ private void ensureLocalIndexTableCreated(byte[] physicalTableName, Map<String,Object> tableProps, List<Pair<byte[],Map<String,Object>>> families, byte[][] splits, long timestamp) throws SQLException {
{code}
Minor, but would you mind to only call MetaDataUtil.isMultiTenant(m, kvBuilder, ptr) once here by just putting an if (localIndexTable inside the if statement?
{code}
- if (physicalTableName != null && !MetaDataUtil.isMultiTenant(m, kvBuilder, ptr)) {
+ if (physicalTableName != null && !MetaDataUtil.isMultiTenant(m, kvBuilder, ptr) && localIndexTable) {
+ ensureLocalIndexTableCreated(tableName, tableProps, families, splits, MetaDataUtil.getClientTimeStamp(m));
+ } else if (physicalTableName != null && !MetaDataUtil.isMultiTenant(m, kvBuilder, ptr)) {
{code}
Add a call to ensureLocalIndexTableCreated in this else if in ConnectionQueryServicesImpl too, as we'd want to create both a "global" index table and a "local" index table up front for views on multi-tenant table. We try to do all the HBase DDL operations up front for a multi-tenant table, and it'd be ok to have both global and local indexes for a view on a multi-tenant table:
{code}
} else if (tableType == PTableType.TABLE && MetaDataUtil.isMultiTenant(m, kvBuilder, ptr)) { // Create view index table up front for multi tenant tables
{code}
was (Author: jamestaylor):
Thanks, [~rajesh23]. Here's some feedback:
You won't need the following change, as this is a facility to pre-split a table. We won't allow a local index to be pre-split or salted -this is an error case we catch earlier:
- splits = SchemaUtil.processSplits(splits, pkColumns, saltBucketNum, connection.getQueryServices().getProps().getBoolean(
+ if (parent != null && tableType == PTableType.INDEX && indexType == IndexType.LOCAL) {
+ tableProps.put(MetaDataUtil.PARENT_TABLE_KEY, parent.getPhysicalName().getString());
+ tableProps.put(MetaDataUtil.IS_LOCAL_INDEX_TABLE_PROP_NAME, Boolean.TRUE);
+ splits = getSplitKeys(connection.getQueryServices().getAllTableRegions(parent.getPhysicalName().getBytes()));
+ } else {
+ splits = SchemaUtil.processSplits(splits, pkColumns, saltBucketNum, connection.getQueryServices().getProps().getBoolean(
QueryServices.ROW_KEY_ORDER_SALTED_TABLE_ATTRIB, QueryServicesOptions.DEFAULT_ROW_KEY_ORDER_SALTED_TABLE));
+ }
The new function you added can be simplified, as we don't need to calculate the MAX_FILE_SIZE as a percentage of the data table MAX_FILE_SIZE. We do this for global indexes, as they're typically smaller and you'd want to split them when they're smaller to attempt to have the same number of index regions as data regions (you get better parallelization in that case). For local indexes, we don't need to do this.
+ private void ensureLocalIndexTableCreated(byte[] physicalTableName, Map<String,Object> tableProps, List<Pair<byte[],Map<String,Object>>> families, byte[][] splits, long timestamp) throws SQLException {
Minor, but would you mind to only call MetaDataUtil.isMultiTenant(m, kvBuilder, ptr) once here by just putting an if (localIndexTable inside the if statement?
- if (physicalTableName != null && !MetaDataUtil.isMultiTenant(m, kvBuilder, ptr)) {
+ if (physicalTableName != null && !MetaDataUtil.isMultiTenant(m, kvBuilder, ptr) && localIndexTable) {
+ ensureLocalIndexTableCreated(tableName, tableProps, families, splits, MetaDataUtil.getClientTimeStamp(m));
+ } else if (physicalTableName != null && !MetaDataUtil.isMultiTenant(m, kvBuilder, ptr)) {
Add a call to ensureLocalIndexTableCreated in this else if in ConnectionQueryServicesImpl too, as we'd want to create both a "global" index table and a "local" index table up front for views on multi-tenant table. We try to do all the HBase DDL operations up front for a multi-tenant table, and it'd be ok to have both global and local indexes for a view on a multi-tenant table:
} else if (tableType == PTableType.TABLE && MetaDataUtil.isMultiTenant(m, kvBuilder, ptr)) { // Create view index table up front for multi tenant tables
> create local index table with the same split keys of user table
> ---------------------------------------------------------------
>
> Key: PHOENIX-935
> URL: https://issues.apache.org/jira/browse/PHOENIX-935
> Project: Phoenix
> Issue Type: Sub-task
> Reporter: rajeshbabu
> Assignee: rajeshbabu
> Attachments: PHOENIX-935.patch
>
>
> For co-locating user table regions and index table regions, the regions key ranges should be same. So when user specify to create local index, internally we can create the index table with the same split keys of user table.
--
This message was sent by Atlassian JIRA
(v6.2#6252)