You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@phoenix.apache.org by "Viraj Jasani (Jira)" <ji...@apache.org> on 2025/11/03 20:41:00 UTC
[jira] [Updated] (PHOENIX-7724) CompactionScanner PTable usage should use getTable()
[ https://issues.apache.org/jira/browse/PHOENIX-7724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Viraj Jasani updated PHOENIX-7724:
----------------------------------
Description:
Phoenix implements it's own version of major and minor compactions using CompactionScanner to avoid data integrity issues at the row version retentions, rather than operate at the column family level.
CompactionScanner is initialized with PTable object for the given table. We use
conn.getTableNoCache() to retrieve the PTable for both minor and major compactions, which results in rpc call to SYSTEM.CATALOG regardless of whether the regionserver has the PTable object in it's cache.
With large amount of data and higher num of regions, compaction occurrence would be more frequent. It has tendency to generate large amount of rpc calls at some point.
To avoid this, the purpose of this Jira is to use conn.getTable(), which uses client side cache if PTable is already available to prevent rpc call to syscat.
Sample error logs if compaction cannot reach out to syscat:
{code:java}
2025-11-03 19:43:00,226 ERROR [gionserver/regionserver-1:60020-shortCompactions-1] coprocessor.UngroupedAggregateRegionObserver - Unable to modify compaction scanner to retain deleted cells for a table with disabled Index; TEST1
org.apache.phoenix.exception.PhoenixIOException: SYSTEM.CATALOG is disabled.
at org.apache.phoenix.util.ClientUtil.parseServerException(ClientUtil.java:72)
at org.apache.phoenix.query.ConnectionQueryServicesImpl.metaDataCoprocessorExec(ConnectionQueryServicesImpl.java:2412)
at org.apache.phoenix.query.ConnectionQueryServicesImpl.metaDataCoprocessorExec(ConnectionQueryServicesImpl.java:2358)
at org.apache.phoenix.query.ConnectionQueryServicesImpl.getTable(ConnectionQueryServicesImpl.java:2658)
at org.apache.phoenix.query.DelegateConnectionQueryServices.getTable(DelegateConnectionQueryServices.java:152)
at org.apache.phoenix.schema.MetaDataClient.updateCache(MetaDataClient.java:605)
at org.apache.phoenix.schema.MetaDataClient.updateCache(MetaDataClient.java:481)
at org.apache.phoenix.jdbc.PhoenixConnection.getTableNoCache(PhoenixConnection.java:654)
at org.apache.phoenix.jdbc.PhoenixConnection.getTableNoCache(PhoenixConnection.java:663)
at org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver$4.run(UngroupedAggregateRegionObserver.java:642)
at org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver$4.run(UngroupedAggregateRegionObserver.java:625){code}
was:
Phoenix implements it's own version of major and minor compactions using CompactionScanner to avoid data integrity issues at the row version retentions, rather than operate at the column family level.
CompactionScanner is initialized with PTable object for the given table. We use
conn.getTableNoCache() to retrieve the PTable for both minor and major compactions, which results in rpc call to SYSTEM.CATALOG regardless of whether the regionserver has the PTable object in it's cache.
With large amount of data and higher num of regions, compaction occurrence would be more frequent. It has tendency to generate large amount of rpc calls at some point.
To avoid this, the purpose of this Jira is to use conn.getTable(), which uses client side cache if PTable is already available to prevent rpc call to syscat.
> CompactionScanner PTable usage should use getTable()
> ----------------------------------------------------
>
> Key: PHOENIX-7724
> URL: https://issues.apache.org/jira/browse/PHOENIX-7724
> Project: Phoenix
> Issue Type: Improvement
> Reporter: Viraj Jasani
> Priority: Major
>
> Phoenix implements it's own version of major and minor compactions using CompactionScanner to avoid data integrity issues at the row version retentions, rather than operate at the column family level.
> CompactionScanner is initialized with PTable object for the given table. We use
> conn.getTableNoCache() to retrieve the PTable for both minor and major compactions, which results in rpc call to SYSTEM.CATALOG regardless of whether the regionserver has the PTable object in it's cache.
> With large amount of data and higher num of regions, compaction occurrence would be more frequent. It has tendency to generate large amount of rpc calls at some point.
>
> To avoid this, the purpose of this Jira is to use conn.getTable(), which uses client side cache if PTable is already available to prevent rpc call to syscat.
>
> Sample error logs if compaction cannot reach out to syscat:
> {code:java}
> 2025-11-03 19:43:00,226 ERROR [gionserver/regionserver-1:60020-shortCompactions-1] coprocessor.UngroupedAggregateRegionObserver - Unable to modify compaction scanner to retain deleted cells for a table with disabled Index; TEST1
> org.apache.phoenix.exception.PhoenixIOException: SYSTEM.CATALOG is disabled.
> at org.apache.phoenix.util.ClientUtil.parseServerException(ClientUtil.java:72)
> at org.apache.phoenix.query.ConnectionQueryServicesImpl.metaDataCoprocessorExec(ConnectionQueryServicesImpl.java:2412)
> at org.apache.phoenix.query.ConnectionQueryServicesImpl.metaDataCoprocessorExec(ConnectionQueryServicesImpl.java:2358)
> at org.apache.phoenix.query.ConnectionQueryServicesImpl.getTable(ConnectionQueryServicesImpl.java:2658)
> at org.apache.phoenix.query.DelegateConnectionQueryServices.getTable(DelegateConnectionQueryServices.java:152)
> at org.apache.phoenix.schema.MetaDataClient.updateCache(MetaDataClient.java:605)
> at org.apache.phoenix.schema.MetaDataClient.updateCache(MetaDataClient.java:481)
> at org.apache.phoenix.jdbc.PhoenixConnection.getTableNoCache(PhoenixConnection.java:654)
> at org.apache.phoenix.jdbc.PhoenixConnection.getTableNoCache(PhoenixConnection.java:663)
> at org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver$4.run(UngroupedAggregateRegionObserver.java:642)
> at org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver$4.run(UngroupedAggregateRegionObserver.java:625){code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)