You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@carbondata.apache.org by "Yadong Qi (JIRA)" <ji...@apache.org> on 2017/04/27 07:47:04 UTC
[jira] [Created] (CARBONDATA-998) Don't request executors when we
use carbon distribution
Yadong Qi created CARBONDATA-998:
------------------------------------
Summary: Don't request executors when we use carbon distribution
Key: CARBONDATA-998
URL: https://issues.apache.org/jira/browse/CARBONDATA-998
Project: CarbonData
Issue Type: Improvement
Components: spark-integration
Affects Versions: 1.1.0
Reporter: Yadong Qi
Assignee: Yadong Qi
In the current implementation, carbon request executors if it need more executors. This brings the following questions:
1. Carbon interferes with spark dispatch to make request/remove executors confusedly.
2. Carbon will involk DistributionUtil.getDistinctNodesList and sleep some times to make sure the executors are assigned, it wastes a lot of time.
We'd better make the data scan and schedule independently.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)