You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@carbondata.apache.org by "Yadong Qi (JIRA)" <ji...@apache.org> on 2017/04/27 07:47:04 UTC

[jira] [Created] (CARBONDATA-998) Don't request executors when we use carbon distribution

Yadong Qi created CARBONDATA-998:
------------------------------------

             Summary: Don't request executors when we use carbon distribution
                 Key: CARBONDATA-998
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-998
             Project: CarbonData
          Issue Type: Improvement
          Components: spark-integration
    Affects Versions: 1.1.0
            Reporter: Yadong Qi
            Assignee: Yadong Qi


In the current implementation, carbon request executors if it need more executors. This brings the following questions:
1. Carbon interferes with spark dispatch to make request/remove executors confusedly.
2. Carbon will involk DistributionUtil.getDistinctNodesList and sleep some times to make sure the executors are assigned, it wastes a lot of time.

We'd better make the data scan and schedule independently.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)