You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2019/01/15 01:45:00 UTC
[jira] [Commented] (SAMZA-2059) Create the configuration in
coordinator stream for standalone.
[ https://issues.apache.org/jira/browse/SAMZA-2059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16742656#comment-16742656 ]
ASF GitHub Bot commented on SAMZA-2059:
---------------------------------------
GitHub user shanthoosh opened a pull request:
https://github.com/apache/samza/pull/879
SAMZA-2059: Persist configuration in coordinator stream for standalone.
Prior to Samza 1.0, users plugged in the properties of an I/O system through a configuration file. Samza employed rewriters in the user-defined order to compute the configuration of a job.
Post Samza 1.0, we introduced new abstractions viz` StreamDescriptor` and `SystemDescriptor` in samza, with the purpose of performing configuration expansion for predefined systems at run-time.
Configuration computed at run-time is not persisted at a centralized storage in samza-standalone. This breaks the functionality of the tools viz checkpoint-tool, coordinator-stream-writer, etc in samza standalone. This patch addresses this problem by storing the configuration in coordinator stream for standalone.
In the follow up PR's:
1. We'll switch from zookeeper to coordinator-stream as JobModel storage layer in standalone
2. Samza tools(checkpoint-tool) will be migrated to read the configuration from coordinator stream rather than the disk.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/shanthoosh/samza standalone-coordinator-stream-for-config
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/samza/pull/879.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #879
----
commit 5095eb5dcf447646494eed3cf2b3e9ea8fcf7580
Author: Shanthoosh Venkataraman <sp...@...>
Date: 2019-01-11T21:51:52Z
SAMZA-2059: Storing configuration in coordinator stream for standalone.
----
> Create the configuration in coordinator stream for standalone.
> --------------------------------------------------------------
>
> Key: SAMZA-2059
> URL: https://issues.apache.org/jira/browse/SAMZA-2059
> Project: Samza
> Issue Type: Improvement
> Reporter: Shanthoosh Venkataraman
> Assignee: Shanthoosh Venkataraman
> Priority: Major
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)