You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2018/09/28 00:11:00 UTC
[jira] [Commented] (SAMZA-1714) Creating shared context factory for
shared context objects
[ https://issues.apache.org/jira/browse/SAMZA-1714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16631204#comment-16631204 ]
ASF GitHub Bot commented on SAMZA-1714:
---------------------------------------
GitHub user cameronlee314 opened a pull request:
https://github.com/apache/samza/pull/672
SAMZA-1714: Creating shared context factory for shared context objects
This includes changes in https://github.com/apache/samza/pull/626.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/cameronlee314/samza shared_context_impl
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/samza/pull/672.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #672
----
commit b6c1588f39e0ad9ccff609c3f6bdb174a7109055
Author: Cameron Lee <ca...@...>
Date: 2018-09-07T18:10:26Z
SAMZA-1858: Public APIs for shared context
commit e4cc56d5f66049c7b7d2a2624d35ebe278920fda
Author: Cameron Lee <ca...@...>
Date: 2018-09-07T18:40:38Z
fixing ContextImpl and adding tests
commit 6359c8a8268fc1819eec2457e887f36c0e6766fc
Author: Cameron Lee <ca...@...>
Date: 2018-09-07T19:32:28Z
fixing ApplicationDescriptor link in javadoc and using consistent style for instance var names
commit 71122137503717d19861cdee67ffbb36b7350313
Author: Cameron Lee <ca...@...>
Date: 2018-09-07T22:01:30Z
adding SamzaContextImpl and SamzaContainerContext
commit fc8b68b606f7c6b5494282fbaae07bdb269dc6ac
Author: Cameron Lee <ca...@...>
Date: 2018-09-07T22:23:26Z
using SamzaContainerContext instead of SamzaContext for ApplicationDefinedContainerContextFactory
commit 22af79c94f75820a36bdec99000e3f543f8569fb
Author: Cameron Lee <ca...@...>
Date: 2018-09-08T00:45:47Z
removing underscores from instance vars for tests
commit 9a2e5bd2fbfd15ee6b25589e227ee554b4f9915b
Author: Cameron Lee <ca...@...>
Date: 2018-09-10T21:13:20Z
wip: needs unit tests
commit 2f349847d88be1e2d541caa72b989757fb237464
Author: Cameron Lee <ca...@...>
Date: 2018-09-12T00:53:04Z
Merge branch 'master' into shared_context_apis
commit 31de93d030921a5c1a0895e7eac23f3b0ac61fd3
Author: Cameron Lee <ca...@...>
Date: 2018-09-12T22:00:26Z
wip: still need to move task and container model
commit 67c3608d00fb70aa7124c0c671ba281e4dbc5a57
Author: Cameron Lee <ca...@...>
Date: 2018-09-18T00:25:18Z
SAMZA-1880: Rename non-metrics classes which use Timer in their name
commit db4fad6debe101f5103537f7c942011505ea2829
Author: Cameron Lee <ca...@...>
Date: 2018-09-18T00:46:18Z
minor style check update
commit b63e032c2cbb8e1eadbc7bd6acb30bd431272d54
Author: Cameron Lee <ca...@...>
Date: 2018-09-18T20:44:29Z
simplify terminology: rename Scheduling to Scheduled, rename KeyScheduler to Scheduler
commit 970830c491ef90fd675d5cdd8a6134c3950c4b34
Author: Cameron Lee <ca...@...>
Date: 2018-09-19T20:19:57Z
Merge branch 'master' into rename_timer
commit 2af9244509da79c35b3b33c18b6b07d2786aae22
Author: Cameron Lee <ca...@...>
Date: 2018-09-19T20:21:38Z
Merge branch 'rename_timer' into shared_context_apis
commit 7c774f3c929857e406be3e92f31d11dc2fd603c3
Author: Cameron Lee <ca...@...>
Date: 2018-09-19T21:33:18Z
add models to contexts, merge with refactorings
commit 80b01be16fb0998d0dcdd108339c1f905b0e1b2a
Author: Cameron Lee <ca...@...>
Date: 2018-09-19T21:53:50Z
Merge branch 'shared_context_apis' into shared_context_impl
commit 002148f679cacfb9cce9ff6f10c09b6e3489b18a
Author: Cameron Lee <ca...@...>
Date: 2018-09-20T21:40:17Z
update CallbackScheduler doc
commit 2406c76e4d0203b60a366eea9d3f163d06154811
Author: Cameron Lee <ca...@...>
Date: 2018-09-21T00:23:48Z
merge with timer renaming; updating tests with new API
commit aa0c19b03102bd3750482f5f90f4b6bbb1e69f92
Author: Cameron Lee <ca...@...>
Date: 2018-09-25T17:15:10Z
wip
commit 36aa4a4a1fb1584609a937a6391260284e59792f
Author: Cameron Lee <ca...@...>
Date: 2018-09-26T21:07:50Z
update docs, add null checks
commit ff2a863dcacd06f98436f4d3b2ea9d9d593628c9
Author: Cameron Lee <ca...@...>
Date: 2018-09-26T21:17:41Z
Merge branch 'master' into shared_context_apis
commit 5aaaf1f4bc1c87478b98a47c7424986143e38a07
Author: Cameron Lee <ca...@...>
Date: 2018-09-26T23:04:26Z
removing wildcard imports, merge with timer renaming
commit da5fa79d0bf5acd07f7051b26e4a0bf9d6187676
Author: Cameron Lee <ca...@...>
Date: 2018-09-26T23:25:55Z
Merge branch 'shared_context_apis' into shared_context_impl
commit 906412a6390198b3b0a49147dedf56c2877450da
Author: Cameron Lee <ca...@...>
Date: 2018-09-28T00:05:47Z
deleting old classes, updating test imports, fixing compile issues after merge
----
> Creating shared context factory for shared context objects
> ----------------------------------------------------------
>
> Key: SAMZA-1714
> URL: https://issues.apache.org/jira/browse/SAMZA-1714
> Project: Samza
> Issue Type: Improvement
> Reporter: Yi Pan (Data Infrastructure)
> Assignee: Cameron Lee
> Priority: Major
>
> Currently, ContextManager only provides a single scope, single object shared context in the high-level API:
> {code:java}
> public interface ContextManager {
> public void init(Config config, TaskContext context);
> }{code}
> It is not sufficient to address the following requirements:
> 1) different scope of shared objects (i.e. operator-scope, task-scope, container-scope)
> 2) different shared objects instantiated and shared in the same scope (i.e. user program has task-level shared objects for remote-http-client vs in-mem cache that needs to managed independently)
> An initial idea to implement the SharedContextFactory is the following:
> {code}
> public interface SharedContextFactory {
> <T extends SharedContext> T createSharedContext(Config config, Scope scope, final SystemContext sysContext, String id);
> }
> public interface SharedContext {
> void init(Config, final SystemContext sysContext);
> void close();
> }
> public interface SystemContext {
> <T extends SharedContext> T getSharedContext(Scope scope, String id);
> }
> public interface SamzaContainerContext extends SystemContext {
> }
> public interface TaskContext extends SystemContext {
> }
> {code}
> Note that this is also needed to serialize operator functions used in high-level API that depends on non-serializable third-party libraries (e.g. Calcite runtime objects used in Samza SQL).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)