You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@phoenix.apache.org by "Istvan Toth (Jira)" <ji...@apache.org> on 2023/05/04 05:57:00 UTC

[jira] [Created] (PHOENIX-6946) IndexTool wastes MR resources

Istvan Toth created PHOENIX-6946:
------------------------------------

             Summary: IndexTool wastes MR resources
                 Key: PHOENIX-6946
                 URL: https://issues.apache.org/jira/browse/PHOENIX-6946
             Project: Phoenix
          Issue Type: Bug
          Components: core
            Reporter: Istvan Toth


For the new style indexing, IndexTool does little more than generate a custom scan opreration per guidepost, send it to the RS, then wait for the result.

It is idling 99.99% of the time, yet it ties up a lot of (usually all) Yarn resources.

We should be able to do this without MR, or at least consolidate this into a single mapper job (in which case we'd simply use the MR framework as a job scheduler)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)