You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@bigtop.apache.org by "Kevin W Monroe (JIRA)" <ji...@apache.org> on 2017/06/02 18:41:04 UTC

[jira] [Created] (BIGTOP-2795) spark charm: fix sparkpi and rework start/stop logic

Kevin W Monroe created BIGTOP-2795:
--------------------------------------

             Summary: spark charm: fix sparkpi and rework start/stop logic
                 Key: BIGTOP-2795
                 URL: https://issues.apache.org/jira/browse/BIGTOP-2795
             Project: Bigtop
          Issue Type: Improvement
            Reporter: Kevin W Monroe
            Assignee: Kevin W Monroe
            Priority: Minor
             Fix For: 1.2.1


The current spark charm has a couple problems:

- The sample workload (delivered as ~/sparkpi.sh) hard codes the spark-examples.jar path, and it's incorrect for spark2.
- Expose the spark worker ui port (8081) when appropriate.
- We only need to open the master ui port (8080) if we're in standalone mode.  There is no master process running in yarn or local modes.
- We should only wait for master recovery if we have peers. It's important to wait for recovery before starting workers in a multi-node cluster, but for single-node, this is a waste of time.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)