[jira] [Commented] (SAMZA-1788) Introduce LocationIdProvider abstraction

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SAMZA-1788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16559090#comment-16559090 ] 

ASF GitHub Bot commented on SAMZA-1788:
---------------------------------------

GitHub user shanthoosh opened a pull request:

    https://github.com/apache/samza/pull/585

    SAMZA-1788: Add LocationIdProvider abstraction.

    Currently in standalone, by default hostName of the standalone processor is used as LocationId. However, for containerized environments like azure cloud, kubernetes this defaulting does not work. Standalone processors can be launched from different kubernetes container on a physical machine(where each docker container has different locatliyID than other docker container within same machine). 
    
    To solve this problem, we introduce locationID abstraction to allow users to plugin a uniqueId identifying the execution environment of the processor.
    
    In containerized environments, LocationId is a composite key of multiple fields: (sliceId, containerId, hostname) By default hostname will be used as LocationId(if not configured by the user).
    
    All the processors of an application registered from an locationID should be able to share(read/write) their local state stores. Any custom LocationIdProvider is expected to honor this contract when generating the locationID.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/shanthoosh/samza add_location_id_interface

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/samza/pull/585.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #585
    
----
commit a9ba9bbf3e8e7cdb96f69fd05c56eded015e3c65
Author: Shanthoosh Venkataraman <sp...@...>
Date:   2018-07-26T23:54:31Z

    SAMZA-1788: Introduce LocationIdProvider abstraction.

----


> Introduce LocationIdProvider abstraction
> ----------------------------------------
>
>                 Key: SAMZA-1788
>                 URL: https://issues.apache.org/jira/browse/SAMZA-1788
>             Project: Samza
>          Issue Type: Task
>            Reporter: Shanthoosh Venkataraman
>            Assignee: Shanthoosh Venkataraman
>            Priority: Major
>
> Currently in standalone, hostName of the standalone processor is used as LocationId. However, for containerized environments like azure cloud, kubernetes this defaulting does  not work. Standalone processors can be launched from different kubernetes container on a physical machine(where each docker container has different locatliyID than other docker container within same machine). 
> To solve this problem, we introduce locationID abstraction to support plugging in uniqueId identifying the execution environment of the processor. In case of containerized environments, LocationId is a combination of multiple fields (sliceId, containerId, hostname) instead of simple physical hostname. By default hostname will be used as LocationId(if not configured by the user).
> A new abstraction LocationIdProvider will be introduced to generate locationId for a physical execution environment. All the processors of an application registered from an locationID should be able to share(read/write) their local state stores. Any custom LocationIdProvider is expected to honor this contract when generating the locationID.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)