You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@samza.apache.org by Navina Ramesh <nr...@linkedin.com> on 2015/07/15 20:24:29 UTC
Re: Review Request 33419: SAMZA-625: Provide tool to consume
changelog and materialize a state store
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33419/#review91785
-----------------------------------------------------------
Ship it!
Looks good to me. Very useful tool, Yan. Nice work!
samza-core/src/main/java/org/apache/samza/config/JavaStorageConfig.java (line 35)
<https://reviews.apache.org/r/33419/#comment145436>
nit: use static strings , esp. when it is repeated. (like ".factory")
samza-core/src/main/java/org/apache/samza/config/JavaSystemConfig.java (line 42)
<https://reviews.apache.org/r/33419/#comment145435>
nit: SYSTEM_FACTORY can be defined in this class rather than from the scala class. Perhaps a future work of cleanup can do this as well.
- Navina Ramesh
On June 21, 2015, 6:10 a.m., Yan Fang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33419/
> -----------------------------------------------------------
>
> (Updated June 21, 2015, 6:10 a.m.)
>
>
> Review request for samza.
>
>
> Bugs: SAMZA-625
> https://issues.apache.org/jira/browse/SAMZA-625
>
>
> Repository: samza
>
>
> Description
> -------
>
> Implemented in Java.
>
> * modified build.gradle to have the gradle compile scala first. Because some jave code has dependencies to Scala code
> * change the state store name by removing the space ( in TaskManager )
> * add scala java conversion method in Util because some classes only accept scala map
> * add java version of some configs
> * remove duplicated config in samza-log4j
> * add StorageRevoery class, which does most of the recoverying job. The logic mimics what happens in SamzaContainer.
> * add StateStorageTool, for the commandline usage
> * unit tests
> * docs
>
>
> Diffs
> -----
>
> checkstyle/import-control.xml 3374f0c
> docs/learn/documentation/versioned/container/state-management.md 79067bb
> samza-core/src/main/java/org/apache/samza/config/JavaStorageConfig.java PRE-CREATION
> samza-core/src/main/java/org/apache/samza/config/JavaSystemConfig.java PRE-CREATION
> samza-core/src/main/java/org/apache/samza/storage/StateStorageTool.java PRE-CREATION
> samza-core/src/main/java/org/apache/samza/storage/StorageRecovery.java PRE-CREATION
> samza-core/src/main/scala/org/apache/samza/storage/TaskStorageManager.scala aeba61a
> samza-core/src/main/scala/org/apache/samza/util/Util.scala 2feb65b
> samza-core/src/test/java/org/apache/samza/config/TestJavaStorageConfig.java PRE-CREATION
> samza-core/src/test/java/org/apache/samza/config/TestJavaSystemConfig.java PRE-CREATION
> samza-core/src/test/java/org/apache/samza/storage/MockStorageEngine.java PRE-CREATION
> samza-core/src/test/java/org/apache/samza/storage/MockStorageEngineFactory.java PRE-CREATION
> samza-core/src/test/java/org/apache/samza/storage/MockSystemConsumer.java PRE-CREATION
> samza-core/src/test/java/org/apache/samza/storage/MockSystemFactory.java PRE-CREATION
> samza-core/src/test/java/org/apache/samza/storage/TestStorageRecovery.java PRE-CREATION
> samza-log4j/src/main/java/org/apache/samza/config/Log4jSystemConfig.java d5e24f2
> samza-shell/src/main/bash/state-storage-tool.sh PRE-CREATION
>
> Diff: https://reviews.apache.org/r/33419/diff/
>
>
> Testing
> -------
>
> tested with multiple partitions and multiple stores recovery.
>
>
> Thanks,
>
> Yan Fang
>
>