You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@phoenix.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2022/04/26 06:56:00 UTC
[jira] [Commented] (PHOENIX-6694) Share the query plan generated in data source reader in partition readers to avoid the unnecessary touch basing system tables and meta table in all the workers.
[ https://issues.apache.org/jira/browse/PHOENIX-6694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17527937#comment-17527937 ]
ASF GitHub Bot commented on PHOENIX-6694:
-----------------------------------------
chrajeshbabu opened a new pull request, #78:
URL: https://github.com/apache/phoenix-connectors/pull/78
…partition readers to avoid the unnecessary touch basing system tables and meta table in all the workers.
> Share the query plan generated in data source reader in partition readers to avoid the unnecessary touch basing system tables and meta table in all the workers.
> ----------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: PHOENIX-6694
> URL: https://issues.apache.org/jira/browse/PHOENIX-6694
> Project: Phoenix
> Issue Type: Bug
> Components: spark-connector
> Reporter: Rajeshbabu Chintaguntla
> Assignee: Rajeshbabu Chintaguntla
> Priority: Major
>
> Currently we are preparing the query plan in both data source and partition readers which is creating new connection in each worker and job initialisation which unnecessarily touch basing all both system catalog table, system stats table as well as meta. When there are jobs with millions of parallel workers hotspot the region servers holding the meta and system catalog as well system stats table. So if we share the same query plan between the workers which can avoid the hotspot.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)