You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by "Chris Riccomini (JIRA)" <ji...@apache.org> on 2015/02/04 03:58:35 UTC
[jira] [Updated] (SAMZA-516) Support standalone Samza jobs
[ https://issues.apache.org/jira/browse/SAMZA-516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Riccomini updated SAMZA-516:
----------------------------------
Attachment: DESIGN-SAMZA-516-2.pdf
DESIGN-SAMZA-516-2.md
Attaching design doc with a straw man proposed solution. The proposal is thinking through the 1:1 process:container ZooKeeper managed solution. This implementation seems a bit complicated. There just seem to be a lot of mis-matches between the way Samza was designed, and how we're trying to shoe-horn it into a ZooKeeper managed model.
Have a look. Feedback welcome.
> Support standalone Samza jobs
> -----------------------------
>
> Key: SAMZA-516
> URL: https://issues.apache.org/jira/browse/SAMZA-516
> Project: Samza
> Issue Type: Bug
> Components: container
> Affects Versions: 0.9.0
> Reporter: Chris Riccomini
> Assignee: Chris Riccomini
> Labels: design, project
> Attachments: DESIGN-SAMZA-516-0.md, DESIGN-SAMZA-516-0.pdf, DESIGN-SAMZA-516-1.md, DESIGN-SAMZA-516-1.pdf, DESIGN-SAMZA-516-2.md, DESIGN-SAMZA-516-2.pdf
>
>
> Samza currently supports two modes of operation out of the box: local and YARN. With local mode, a single Java process starts the JobCoordinator, creates a single container, and executes it locally. All partitions are procesed within this container. With YARN, a YARN grid is required to execute the Samza job. In addition, SAMZA-375 introduces a patch to run Samza in Mesos.
> There have been several requests lately to be able to run Samza jobs without any resource manager (YARN, Mesos, etc), but still run it in a distributed fashion.
> The goal of this ticket is to design and implement a samza-standalone module, which will:
> # Support executing a single Samza job in one or more containers.
> # Support failover, in cases where a machine is lost.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)