[jira] [Commented] (SAMZA-1773) Side inputs for local stores

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SAMZA-1773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16548817#comment-16548817 ] 

ASF GitHub Bot commented on SAMZA-1773:
---------------------------------------

GitHub user bharathkk opened a pull request:

    https://github.com/apache/samza/pull/570

    SAMZA-1773: Side inputs for local stores

    @prateekm @vjagadish 
    Please take a look.
    
    I will update the PR with the unit tests for SideInputStorageManager and the util functions.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/bharathkk/samza side-input-v3

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/samza/pull/570.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #570
    
----
commit 5393812b8ac0e67febaa9641e2db89073a17f434
Author: Bharath Kumarasubramanian <bk...@...>
Date:   2018-07-10T00:49:51Z

    Initial prototype for side input

commit 9ebdc1a15d50ae97519a4ba04a38677a34342056
Author: Bharath Kumarasubramanian <bk...@...>
Date:   2018-07-12T23:36:24Z

    Consolidated the side input logic into TaskStorageManager

commit fa793cb92b4e71fea2e6d21dfd8e23966f2f9ffd
Author: Bharath Kumarasubramanian <bk...@...>
Date:   2018-07-13T00:38:12Z

    Include the initialization of starting offsets for side input stores

commit 34675938cbf4e43c374038fd920fef08617d1d56
Author: Bharath Kumarasubramanian <bk...@...>
Date:   2018-07-13T23:01:41Z

    Alternate prototype for side input.

commit c6fd64fc0b45e5b7148166f19305d2027b75fa86
Author: Bharath Kumarasubramanian <bk...@...>
Date:   2018-07-13T23:15:57Z

    Extracted getting the start offset in TaskInstance to a method

commit fbc808f78308ba611ae199fb95e34f236cafab03
Author: Bharath Kumarasubramanian <bk...@...>
Date:   2018-07-14T01:01:48Z

    Merge the task context impl changes and fix some typos

commit c025eb6b62f5a886242e59c7ba48b93a9f4ff48b
Author: Bharath Kumarasubramanian <bk...@...>
Date:   2018-07-16T17:57:55Z

    Fix checkstyle issues and add copyrights notes

commit e4f66b32401e08e42db04ef24b884f76970de164
Author: Bharath Kumarasubramanian <bk...@...>
Date:   2018-07-16T21:09:53Z

    Fix unit tests and apply persistence store check for side input stores as well

commit 9993f76bc93b5e26541b60c67f90d8b08a0b04bc
Author: Bharath Kumarasubramanian <bk...@...>
Date:   2018-07-17T21:28:15Z

    Make the side input processor factory store level configuration and minor changes to validation

commit cf14cd54df920cc8c9fe6c8de6edc49e7d83c080
Author: Bharath Kumarasubramanian <bk...@...>
Date:   2018-07-19T05:17:22Z

    Addressed Prateek's comments, merged his suggestions and added documentation for the new configs

----


> Side inputs for local stores
> ----------------------------
>
>                 Key: SAMZA-1773
>                 URL: https://issues.apache.org/jira/browse/SAMZA-1773
>             Project: Samza
>          Issue Type: Task
>          Components: kv-store
>            Reporter: Bharath Kumarasubramanian
>            Assignee: Bharath Kumarasubramanian
>            Priority: Major
>
> We want a way to populate the local store for jobs from a secondary data source (e.g hadoop), but is otherwise read-only. Typical use cases of such stores are scoring data for relevance jobs, etc
>  
> The rows (keys) in this dataset are independent of each other, and there's no requirement for the entire dataset to be updated atomically. This means that once the store is bootstrapped, it can be updated in change capture mode in the background while processing continues.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)