You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2017/10/10 00:47:00 UTC
[jira] [Commented] (SAMZA-1424) Extend Samza's StorageEngine APIs
to support Windowed stores
[ https://issues.apache.org/jira/browse/SAMZA-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16197951#comment-16197951 ]
ASF GitHub Bot commented on SAMZA-1424:
---------------------------------------
GitHub user vjagadish1989 opened a pull request:
https://github.com/apache/samza/pull/321
SAMZA-1424 SAMZA-1425 SAMZA-1426 Support serdes and persistent state for windows
Notable changes:
* Made windows durable with support for persistent recoverable stores
* New storage format to support multiple messages in windows (the previous storage format
of storing all messages as a list incurs significant serde overhead)
* Wire TimeSeriesStore with the WindowOperator implementation.
Testing:
* Existing unit tests and integration tests pass with serdes wired-up
* Will follow-up with a PR for hello-samza soon.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/vjagadish1989/samza window-operator-serde
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/samza/pull/321.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #321
----
commit f0509f625a5bd9ba4aaae246e3559a8474c3752e
Author: Jagadish <jv...@linkedin.com>
Date: 2017-10-05T21:59:58Z
fix issues with window store
commit a440e1a56fad92ec63d5cfd8b57548852e8c0517
Author: Jagadish <jv...@linkedin.com>
Date: 2017-10-07T21:14:47Z
Add Serde support for WindowOperator
commit cb2702b5adc663735f0f78edc172c3c30b1006e1
Author: Jagadish <jv...@linkedin.com>
Date: 2017-10-07T21:27:48Z
fix more tests
commit e6534d81dedb6fbc23bdb17274e2c0735e128a67
Author: Jagadish <jv...@linkedin.com>
Date: 2017-10-07T21:32:01Z
merge with master
commit 56ff8fa9d54a06b238884015189b8ac1f7fe6132
Author: Jagadish <jv...@linkedin.com>
Date: 2017-10-08T01:02:19Z
Added wireup of stores to windows
commit 217e3b24a2b49ec5cb3af8dd3b57ad9bfbc15826
Author: Jagadish <jv...@linkedin.com>
Date: 2017-10-08T01:27:47Z
Fix tests to take mock stores
commit c0ad505a8def765a960f30567a0cd9e4a7d858f8
Author: Jagadish <jv...@linkedin.com>
Date: 2017-10-09T21:36:21Z
Fix checkstyle issues. Remove longs from WindowKey
commit 7696f122dd7b8561306cbe8af99c80601ec9b453
Author: Jagadish <jv...@linkedin.com>
Date: 2017-10-09T22:55:17Z
Fix checkstyle issues. Refactor methods for paneId handling
commit cf7c7748571050fed13e9eff4d65b5ad57eb3694
Author: Jagadish <jv...@linkedin.com>
Date: 2017-10-09T23:10:14Z
Fix checkstyle issues. Refactor methods for paneId handling
commit b16e16da72263be28ed4835509a939b8400a0e44
Author: Jagadish <jv...@linkedin.com>
Date: 2017-10-09T23:46:37Z
Add convenience APIs to TimeSeriesStore
commit 4ba3b0ed4ef420844a7c14e19a3c3a72a833eddd
Author: Jagadish <jv...@linkedin.com>
Date: 2017-10-09T23:53:14Z
Add convenience APIs to TimeSeriesStore
commit 1c9c3c64c73658265b32c79bdf4eb1d45a7d982f
Author: Jagadish <jv...@linkedin.com>
Date: 2017-10-09T23:57:41Z
Remove dependency on TestInMemoryStore for classes in samza-core
commit eba5bf6e190ba2702f3843987327e6e32ff1e084
Author: Jagadish <jv...@linkedin.com>
Date: 2017-10-10T00:35:01Z
Add javadocs and checkstyle fixes
commit 09c77ebc5fc945cdf78c80b07f3330859bc46a93
Author: Jagadish <jv...@linkedin.com>
Date: 2017-10-10T00:38:31Z
Undo changes to SystemConsumers.scala
----
> Extend Samza's StorageEngine APIs to support Windowed stores
> ------------------------------------------------------------
>
> Key: SAMZA-1424
> URL: https://issues.apache.org/jira/browse/SAMZA-1424
> Project: Samza
> Issue Type: Sub-task
> Reporter: Jagadish
>
> This will enable using WindowStore with other existing stores we have in Samza: ChangeloggedStore, CachedStore, BatchingStore etc.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)