You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@phoenix.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2019/01/02 21:14:00 UTC
[jira] [Commented] (PHOENIX-5069) Use asynchronous refresh to
provide non-blocking Phoenix Stats Client Cache
[ https://issues.apache.org/jira/browse/PHOENIX-5069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16732430#comment-16732430 ]
ASF GitHub Bot commented on PHOENIX-5069:
-----------------------------------------
Github user karanmehta93 commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/416#discussion_r244856169
--- Diff: phoenix-core/src/main/java/org/apache/phoenix/query/GuidePostsCache.java ---
@@ -80,38 +94,46 @@ public GuidePostsCache(ConnectionQueryServices queryServices, Configuration conf
// Log removals at TRACE for debugging
.removalListener(new PhoenixStatsCacheRemovalListener())
// Automatically load the cache when entries are missing
- .build(new StatsLoader());
+ .build(new PhoenixStatsCacheLoader(new StatsLoaderImpl(), executor));
}
/**
- * {@link CacheLoader} implementation for the Phoenix Table Stats cache.
+ * {@link PhoenixStatsLoader} implementation for the Stats Loader.
*/
- protected class StatsLoader extends CacheLoader<GuidePostsKey, GuidePostsInfo> {
+ protected class StatsLoaderImpl implements PhoenixStatsLoader {
+ @Override
+ public boolean needsLoad() {
+ // Whenever it's called, we try to load stats from stats table
+ // no matter it has been updated or not.
+ return true;
+ }
+
@Override
- public GuidePostsInfo load(GuidePostsKey statsKey) throws Exception {
+ public GuidePostsInfo loadStats(GuidePostsKey statsKey, GuidePostsInfo prevGuidepostInfo) throws Exception {
@SuppressWarnings("deprecation")
- Table statsHTable = queryServices.getTable(SchemaUtil.getPhysicalName(
+ TableName tableName = SchemaUtil.getPhysicalName(
PhoenixDatabaseMetaData.SYSTEM_STATS_NAME_BYTES,
- queryServices.getProps()).getName());
+ queryServices.getProps());
+ Table statsHTable = queryServices.getTable(tableName.getName());
try {
GuidePostsInfo guidePostsInfo = StatisticsUtil.readStatistics(statsHTable, statsKey,
HConstants.LATEST_TIMESTAMP);
traceStatsUpdate(statsKey, guidePostsInfo);
return guidePostsInfo;
} catch (TableNotFoundException e) {
// On a fresh install, stats might not yet be created, don't warn about this.
- logger.debug("Unable to locate Phoenix stats table", e);
- return GuidePostsInfo.NO_GUIDEPOST;
+ logger.debug("Unable to locate Phoenix stats table: " + tableName.toString(), e);
+ return prevGuidepostInfo;
} catch (IOException e) {
- logger.warn("Unable to read from stats table", e);
+ logger.warn("Unable to read from stats table: " + tableName.toString(), e);
// Just cache empty stats. We'll try again after some time anyway.
--- End diff --
nit: remove this comment.
> Use asynchronous refresh to provide non-blocking Phoenix Stats Client Cache
> ---------------------------------------------------------------------------
>
> Key: PHOENIX-5069
> URL: https://issues.apache.org/jira/browse/PHOENIX-5069
> Project: Phoenix
> Issue Type: Improvement
> Reporter: Bin Shi
> Assignee: Bin Shi
> Priority: Major
>
> The current Phoenix Stats Cache uses TTL based eviction policy. A cached entry will expire after a given amount of time (900s by default) passed since the entry's been created. This will lead to cache miss when Compiler/Optimizer fetches stats from cache at the next time. As you can see from the above graph, fetching stats from the cache is a blocking operation — when there is cache miss, it has a round trip over the wire to scan the SYSTEM.STATS Table and to get the latest stats info, rebuild the cache and finally return the stats to the Compiler/Optimizer. Whenever there is a cache miss, this blocking call causes significant performance penalty and see periodic spikes.
> *This Jira suggests to use asynchronous refresh mechanism to provide a non-blocking cache. For details, please see the linked design document below.*
> [~karanmehta93] [~twdsilva@gmail.com] [~dbwong] [~elserj] [~ankit@apache.org] [~sergey soldatov]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)