You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@samza.apache.org by Navina Ramesh <nr...@linkedin.com> on 2015/09/21 11:12:39 UTC
Re: Review Request 33453: SAMZA-557 Reuse local state in
SamzaContainer on clean shutdown
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33453/
-----------------------------------------------------------
(Updated Sept. 21, 2015, 9:12 a.m.)
Review request for samza, Yan Fang, Chris Riccomini, Naveen Somasundaram, and Yi Pan (Data Infrastructure).
Bugs: SAMZA-557
https://issues.apache.org/jira/browse/SAMZA-557
Repository: samza
Description
-------
Added checksum to the Offset file and some unit tests
Added Unit Tests for TaskStorageManager and refactored some code
Changed default to yarn cwd instead of io.tmpDir and refactored code
Diffs
-----
samza-core/src/main/scala/org/apache/samza/config/ShellCommandConfig.scala e94a4735217f59d074510ce1556c8c439e6a72f0
samza-core/src/main/scala/org/apache/samza/container/SamzaContainer.scala ac4793afe1e6868933e750181bee1e27c157b5e6
samza-core/src/main/scala/org/apache/samza/storage/TaskStorageManager.scala f68a7fee24614fce101e91c4f933d9b4e65dda0a
samza-core/src/main/scala/org/apache/samza/util/Util.scala 8a83566ae6139127d7fe04ab42231151227dc479
samza-core/src/test/scala/org/apache/samza/storage/TestTaskStorageManager.scala PRE-CREATION
samza-core/src/test/scala/org/apache/samza/util/TestUtil.scala b75f44060fb8e660e824eaeb9cfdcc9d6fa902e8
samza-kv-rocksdb/src/main/scala/org/apache/samza/storage/kv/RocksDbKeyValueStore.scala 1b44a517129b35affac802929087eaa0061e6b5d
Diff: https://reviews.apache.org/r/33453/diff/
Testing
-------
Tested locally using hello-samza.
Note: you have to set an environment variable LOGGED_STORE_BASE_DIR pointing to the new location to persist the changelog attached stores. Otherwise, it will default to YARN's cwd and will not re-use local state.
Thanks,
Navina Ramesh