You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@phoenix.apache.org by "James Taylor (JIRA)" <ji...@apache.org> on 2017/10/13 05:19:00 UTC
[jira] [Comment Edited] (PHOENIX-4242) Fix Indexer post-compact
hook logging of NPE and TableNotFound
[ https://issues.apache.org/jira/browse/PHOENIX-4242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16203061#comment-16203061 ]
James Taylor edited comment on PHOENIX-4242 at 10/13/17 5:18 AM:
-----------------------------------------------------------------
Here's some initial feedback, but would you mind reviewing this too, [~tdsilva]?
- You'll want to select COLUMN_NAME in addition to COLUMN_FAMILY as it looks like we store the tenant ID there. Then we'll need to create a new PhoenixConnection for the {{PhoenixRuntime.getTableNoCache()}} call whenever the value of COLUMN_NAME changes (creating a connection is not an expensive operation). You'd set the PhoenixRuntime.TenantId connection property so that you'd only be looking for the tables for that tenant. You can collect up as many different TABLE_SCHEM, TABLE_NAME pairs for the same TENANT_ID and query for them in a single shot (i.e. WHERE (%s, %s, %s, %s) IN ((?,?,?,?), (?,?,?,?), ...).
{code}
+ private static final String CHILD_LINK_QUERY =
+ String.format(
+ "SELECT " + COLUMN_FAMILY + " FROM " + SYSTEM_CATALOG
+ + " WHERE (%s, %s, %s, %s) IN ((?,?,?,?))",
+ TENANT_ID, TABLE_SCHEM, TABLE_NAME, LINK_TYPE);
+
+ private static void getAllDescendantViewsRecursive(Connection conn, PName tenantId, PName tableSchem, PName tableName, List<PTable> results) throws SQLException {
+ try (PreparedStatement ps = conn.prepareStatement(CHILD_LINK_QUERY)) {
+ ps.setString(1, tenantId == null ? null : tenantId.getString());
+ ps.setString(2, tableSchem == null ? null : tableSchem.getString());
+ ps.setString(3, tableName.getString());
+ ps.setByte(4, PTable.LinkType.CHILD_TABLE.getSerializedValue());
+ try (ResultSet rs = ps.executeQuery()) {
+ while (rs.next()) {
+ String childTableName = rs.getString(1);
+ PTable childPTable = PhoenixRuntime.getTableNoCache(conn, childTableName);
+ results.add(childPTable);
+ getAllDescendantViewsRecursive(conn, childPTable.getTenantId(),
+ childPTable.getSchemaName(), childPTable.getTableName(), results);
+ }
+ }
+ }
+ }
{code}
- Create a test that requires the above. A good way would be to have the same named table for different tenants. You should only pickup the correct children.
- An important one to follow up on is PHOENIX-4263 to see if our partial index rebuilder handles rebuilding indexes on views.
- [~tdsilva] - did we consider encoding the tenant ID in the COLUMN_FAMILY column so we could keep the COLUMN_NAME null to ensure linking rows would be grouped together? Or an alternative would be to add another nullable column (like CHILD_TENANT_ID) at the end of the SYSTEM CATALOG row key. We may want to consider changing this because there can be many many child linking rows potentially. Maybe file a JIRA for further discussion?
was (Author: jamestaylor):
Here's some initial feedback, but would you mind reviewing this too, [~tdsilva]?
- You'll want to select COLUMN_NAME in addition to COLUMN_FAMILY as it looks like we store the tenant ID there. Then we'll need to create a new PhoenixConnection for the {{PhoenixRuntime.getTableNoCache()}} call whenever the value of COLUMN_NAME changes (creating a connection is not an expensive operation). You'd set the PhoenixRuntime.TenantId connection property so that you'd only be looking for the tables for that tenant. You can collect up as many different TABLE_SCHEM, TABLE_NAME pairs for the same TENANT_ID and query for them in a single shot (i.e. WHERE (%s, %s, %s, %s) IN ((?,?,?,?), (?,?,?,?), ...).
{code}
+ private static final String CHILD_LINK_QUERY =
+ String.format(
+ "SELECT " + COLUMN_FAMILY + " FROM " + SYSTEM_CATALOG
+ + " WHERE (%s, %s, %s, %s) IN ((?,?,?,?))",
+ TENANT_ID, TABLE_SCHEM, TABLE_NAME, LINK_TYPE);
+
+ private static void getAllDescendantViewsRecursive(Connection conn, PName tenantId, PName tableSchem, PName tableName, List<PTable> results) throws SQLException {
+ try (PreparedStatement ps = conn.prepareStatement(CHILD_LINK_QUERY)) {
+ ps.setString(1, tenantId == null ? null : tenantId.getString());
+ ps.setString(2, tableSchem == null ? null : tableSchem.getString());
+ ps.setString(3, tableName.getString());
+ ps.setByte(4, PTable.LinkType.CHILD_TABLE.getSerializedValue());
+ try (ResultSet rs = ps.executeQuery()) {
+ while (rs.next()) {
+ String childTableName = rs.getString(1);
+ PTable childPTable = PhoenixRuntime.getTableNoCache(conn, childTableName);
+ results.add(childPTable);
+ getAllDescendantViewsRecursive(conn, childPTable.getTenantId(),
+ childPTable.getSchemaName(), childPTable.getTableName(), results);
+ }
+ }
+ }
+ }
{code}
- Create a test that requires the above. A good way would be to have the same named table for different tenants. You should only pickup the correct children.
- [~tdsilva] - did we consider encoding the tenant ID in the COLUMN_FAMILY column so we could keep the COLUMN_NAME null to ensure linking rows would be grouped together? Or an alternative would be to add another nullable column (like CHILD_TENANT_ID) at the end of the SYSTEM CATALOG row key. We may want to consider changing this because there can be many many child linking rows potentially. Maybe file a JIRA for further discussion?
> Fix Indexer post-compact hook logging of NPE and TableNotFound
> --------------------------------------------------------------
>
> Key: PHOENIX-4242
> URL: https://issues.apache.org/jira/browse/PHOENIX-4242
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 4.12.0
> Reporter: Vincent Poon
> Assignee: Vincent Poon
> Attachments: PHOENIX-4242.v2.master.patch, PHOENIX-4242.v3.master.patch, PHOENIX-4747.v1.master.patch
>
>
> The post-compact hook in the Indexer seems to log extraneous log messages indicating NPE or TableNotFound. The TableNotFound exceptions seem to indicate actual table names prefixed with MERGE or RESTORE, and sometimes suffixed with a digit, so perhaps these are views or something similar.
> Examples:
> 2017-09-28 13:35:03,118 WARN [ctions-1506410238599] index.Indexer - Unable to permanently disable indexes being partially rebuild for SYSTEM.SEQUENCE
> java.lang.NullPointerException
> 2017-09-28 10:20:56,406 WARN [ctions-1506410238415] index.Indexer - Unable to permanently disable indexes being partially rebuild for MERGE_PLATFORM_ENTITY.PLATFORM_IMMUTABLE_ENTITY_DATA2
> org.apache.phoenix.schema.TableNotFoundException: ERROR 1012 (42M03): Table undefined. tableName=MERGE_PLATFORM_ENTITY.PLATFORM_IMMUTABLE_ENTITY_DATA2
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)