You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@samza.apache.org by Guozhang Wang <wa...@gmail.com> on 2015/05/01 20:43:18 UTC
Review Request 33761: Fix SAMZA-658
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33761/
-----------------------------------------------------------
Review request for samza.
Repository: samza
Description
-------
Add removing logic to the intermediate cached store for kv engine
Diffs
-----
samza-kv/src/main/scala/org/apache/samza/storage/kv/BaseKeyValueStorageEngineFactory.scala b3624e6057ee1a86090f00d2853035b06f63358d
samza-kv/src/main/scala/org/apache/samza/storage/kv/CachedStore.scala 61bb3f6acb080b653f8b11176538549738255acc
samza-kv/src/main/scala/org/apache/samza/storage/kv/KeyValueStorageEngine.scala 3a23daf053f0b8dec3a7ec83a51c9c5527078a3b
Diff: https://reviews.apache.org/r/33761/diff/
Testing
-------
Thanks,
Guozhang Wang
Re: Review Request 33761: Fix SAMZA-658
Posted by "Yi Pan (Data Infrastructure)" <yi...@linkedin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33761/#review82604
-----------------------------------------------------------
Overall looks good. Just some minor comments.
samza-kv/src/main/scala/org/apache/samza/storage/kv/CachedStore.scala
<https://reviews.apache.org/r/33761/#comment133359>
Refer to in-memory KV-store class, we can merge those two in one function: getIter(iter) and make range() and all() referring to the same function like:
def range(from: K, to: K) = {
metrics.ranges.inc
flush()
getIter(store.range(from, to))
}
samza-kv/src/test/scala/org/apache/samza/storage/kv/TestCachedStore.scala
<https://reviews.apache.org/r/33761/#comment133362>
It would be better to test iter.remove() for both range() and all()
- Yi Pan (Data Infrastructure)
On May 4, 2015, 9:07 p.m., Guozhang Wang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33761/
> -----------------------------------------------------------
>
> (Updated May 4, 2015, 9:07 p.m.)
>
>
> Review request for samza.
>
>
> Bugs: SAMZA-658
> https://issues.apache.org/jira/browse/SAMZA-658
>
>
> Repository: samza
>
>
> Description
> -------
>
> Address Yi's comments, add unit test
>
>
> Diffs
> -----
>
> samza-kv/src/main/scala/org/apache/samza/storage/kv/BaseKeyValueStorageEngineFactory.scala b3624e6057ee1a86090f00d2853035b06f63358d
> samza-kv/src/main/scala/org/apache/samza/storage/kv/CachedStore.scala 61bb3f6acb080b653f8b11176538549738255acc
> samza-kv/src/main/scala/org/apache/samza/storage/kv/KeyValueStorageEngine.scala 3a23daf053f0b8dec3a7ec83a51c9c5527078a3b
> samza-kv/src/test/scala/org/apache/samza/storage/kv/MockKeyValueStore.scala PRE-CREATION
> samza-kv/src/test/scala/org/apache/samza/storage/kv/TestCachedStore.scala d03ec925b103ccf3c1561de0461fbc39cbe9d9ca
>
> Diff: https://reviews.apache.org/r/33761/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Guozhang Wang
>
>
Re: Review Request 33761: Fix SAMZA-658
Posted by "Yi Pan (Data Infrastructure)" <yi...@linkedin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33761/#review82777
-----------------------------------------------------------
Ship it!
Ship It!
- Yi Pan (Data Infrastructure)
On May 6, 2015, 11:38 p.m., Guozhang Wang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33761/
> -----------------------------------------------------------
>
> (Updated May 6, 2015, 11:38 p.m.)
>
>
> Review request for samza.
>
>
> Bugs: SAMZA-658
> https://issues.apache.org/jira/browse/SAMZA-658
>
>
> Repository: samza
>
>
> Description
> -------
>
> Address Yi's comments round two
>
>
> Diffs
> -----
>
> samza-kv-inmemory/src/main/scala/org/apache/samza/storage/kv/inmemory/InMemoryKeyValueStore.scala 217333c84c696c0cc1bc3eeabf1c4066a6e89795
> samza-kv/src/main/scala/org/apache/samza/storage/kv/BaseKeyValueStorageEngineFactory.scala b3624e6057ee1a86090f00d2853035b06f63358d
> samza-kv/src/main/scala/org/apache/samza/storage/kv/CachedStore.scala 61bb3f6acb080b653f8b11176538549738255acc
> samza-kv/src/main/scala/org/apache/samza/storage/kv/KeyValueStorageEngine.scala 3a23daf053f0b8dec3a7ec83a51c9c5527078a3b
> samza-kv/src/test/scala/org/apache/samza/storage/kv/MockKeyValueStore.scala PRE-CREATION
> samza-kv/src/test/scala/org/apache/samza/storage/kv/TestCachedStore.scala d03ec925b103ccf3c1561de0461fbc39cbe9d9ca
>
> Diff: https://reviews.apache.org/r/33761/diff/
>
>
> Testing
> -------
>
> unit tests
>
>
> Thanks,
>
> Guozhang Wang
>
>
Re: Review Request 33761: Fix SAMZA-658
Posted by Guozhang Wang <wa...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33761/
-----------------------------------------------------------
(Updated May 7, 2015, 9:11 p.m.)
Review request for samza.
Bugs: SAMZA-658
https://issues.apache.org/jira/browse/SAMZA-658
Repository: samza
Description (updated)
-------
Minor fix
Diffs (updated)
-----
samza-kv-inmemory/src/main/scala/org/apache/samza/storage/kv/inmemory/InMemoryKeyValueStore.scala 23d028b3796e57a0ed16850f9a0784c02914ee1e
samza-kv/src/main/scala/org/apache/samza/storage/kv/BaseKeyValueStorageEngineFactory.scala b3624e6057ee1a86090f00d2853035b06f63358d
samza-kv/src/main/scala/org/apache/samza/storage/kv/CachedStore.scala 479016dff00320fa263a1a18621d1b89897adb8e
samza-kv/src/main/scala/org/apache/samza/storage/kv/KeyValueStorageEngine.scala fc677b26f8ed49d3cb1ec7f46d9d494016f178af
samza-kv/src/test/scala/org/apache/samza/storage/kv/MockKeyValueStore.scala PRE-CREATION
samza-kv/src/test/scala/org/apache/samza/storage/kv/TestCachedStore.scala d03ec925b103ccf3c1561de0461fbc39cbe9d9ca
Diff: https://reviews.apache.org/r/33761/diff/
Testing
-------
unit tests
Thanks,
Guozhang Wang
Re: Review Request 33761: Fix SAMZA-658
Posted by Guozhang Wang <wa...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33761/
-----------------------------------------------------------
(Updated May 7, 2015, 5:07 p.m.)
Review request for samza.
Bugs: SAMZA-658
https://issues.apache.org/jira/browse/SAMZA-658
Repository: samza
Description (updated)
-------
Rebase
Diffs (updated)
-----
samza-kv-inmemory/src/main/scala/org/apache/samza/storage/kv/inmemory/InMemoryKeyValueStore.scala 23d028b3796e57a0ed16850f9a0784c02914ee1e
samza-kv/src/main/scala/org/apache/samza/storage/kv/BaseKeyValueStorageEngineFactory.scala b3624e6057ee1a86090f00d2853035b06f63358d
samza-kv/src/main/scala/org/apache/samza/storage/kv/CachedStore.scala 479016dff00320fa263a1a18621d1b89897adb8e
samza-kv/src/main/scala/org/apache/samza/storage/kv/KeyValueStorageEngine.scala fc677b26f8ed49d3cb1ec7f46d9d494016f178af
samza-kv/src/test/scala/org/apache/samza/storage/kv/MockKeyValueStore.scala PRE-CREATION
samza-kv/src/test/scala/org/apache/samza/storage/kv/TestCachedStore.scala d03ec925b103ccf3c1561de0461fbc39cbe9d9ca
Diff: https://reviews.apache.org/r/33761/diff/
Testing
-------
unit tests
Thanks,
Guozhang Wang
Re: Review Request 33761: Fix SAMZA-658
Posted by Guozhang Wang <wa...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33761/
-----------------------------------------------------------
(Updated May 6, 2015, 11:38 p.m.)
Review request for samza.
Bugs: SAMZA-658
https://issues.apache.org/jira/browse/SAMZA-658
Repository: samza
Description (updated)
-------
Address Yi's comments round two
Diffs (updated)
-----
samza-kv-inmemory/src/main/scala/org/apache/samza/storage/kv/inmemory/InMemoryKeyValueStore.scala 217333c84c696c0cc1bc3eeabf1c4066a6e89795
samza-kv/src/main/scala/org/apache/samza/storage/kv/BaseKeyValueStorageEngineFactory.scala b3624e6057ee1a86090f00d2853035b06f63358d
samza-kv/src/main/scala/org/apache/samza/storage/kv/CachedStore.scala 61bb3f6acb080b653f8b11176538549738255acc
samza-kv/src/main/scala/org/apache/samza/storage/kv/KeyValueStorageEngine.scala 3a23daf053f0b8dec3a7ec83a51c9c5527078a3b
samza-kv/src/test/scala/org/apache/samza/storage/kv/MockKeyValueStore.scala PRE-CREATION
samza-kv/src/test/scala/org/apache/samza/storage/kv/TestCachedStore.scala d03ec925b103ccf3c1561de0461fbc39cbe9d9ca
Diff: https://reviews.apache.org/r/33761/diff/
Testing (updated)
-------
unit tests
Thanks,
Guozhang Wang
Re: Review Request 33761: Fix SAMZA-658
Posted by Guozhang Wang <wa...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33761/
-----------------------------------------------------------
(Updated May 4, 2015, 9:07 p.m.)
Review request for samza.
Bugs: SAMZA-658
https://issues.apache.org/jira/browse/SAMZA-658
Repository: samza
Description
-------
Address Yi's comments, add unit test
Diffs (updated)
-----
samza-kv/src/main/scala/org/apache/samza/storage/kv/BaseKeyValueStorageEngineFactory.scala b3624e6057ee1a86090f00d2853035b06f63358d
samza-kv/src/main/scala/org/apache/samza/storage/kv/CachedStore.scala 61bb3f6acb080b653f8b11176538549738255acc
samza-kv/src/main/scala/org/apache/samza/storage/kv/KeyValueStorageEngine.scala 3a23daf053f0b8dec3a7ec83a51c9c5527078a3b
samza-kv/src/test/scala/org/apache/samza/storage/kv/MockKeyValueStore.scala PRE-CREATION
samza-kv/src/test/scala/org/apache/samza/storage/kv/TestCachedStore.scala d03ec925b103ccf3c1561de0461fbc39cbe9d9ca
Diff: https://reviews.apache.org/r/33761/diff/
Testing
-------
Thanks,
Guozhang Wang
Re: Review Request 33761: Fix SAMZA-658
Posted by Guozhang Wang <wa...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33761/
-----------------------------------------------------------
(Updated May 4, 2015, 8:56 p.m.)
Review request for samza.
Bugs: SAMZA-658
https://issues.apache.org/jira/browse/SAMZA-658
Repository: samza
Description (updated)
-------
Address Yi's comments, add unit test
Diffs (updated)
-----
samza-kv/src/main/scala/org/apache/samza/storage/kv/BaseKeyValueStorageEngineFactory.scala b3624e6057ee1a86090f00d2853035b06f63358d
samza-kv/src/main/scala/org/apache/samza/storage/kv/CachedStore.scala 61bb3f6acb080b653f8b11176538549738255acc
samza-kv/src/main/scala/org/apache/samza/storage/kv/KeyValueStorageEngine.scala 3a23daf053f0b8dec3a7ec83a51c9c5527078a3b
samza-kv/src/test/scala/org/apache/samza/storage/kv/TestCachedStore.scala d03ec925b103ccf3c1561de0461fbc39cbe9d9ca
Diff: https://reviews.apache.org/r/33761/diff/
Testing
-------
Thanks,
Guozhang Wang
Re: Review Request 33761: Fix SAMZA-658
Posted by "Yi Pan (Data Infrastructure)" <yi...@linkedin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33761/#review82361
-----------------------------------------------------------
We would need a unit test to ensure the fix is effective. I.e. create a cached store, creating an iterator, call iterator.remove() and verify that the entry is deleted from the cache.
samza-kv/src/main/scala/org/apache/samza/storage/kv/CachedStore.scala
<https://reviews.apache.org/r/33761/#comment133093>
This iterator needs to be wrapped as well.
samza-kv/src/main/scala/org/apache/samza/storage/kv/KeyValueStorageEngine.scala
<https://reviews.apache.org/r/33761/#comment133094>
nit: If we are cleaning up the coding style, should be consistent w/ wrapperStore.flush() here as well.
samza-kv/src/main/scala/org/apache/samza/storage/kv/KeyValueStorageEngine.scala
<https://reviews.apache.org/r/33761/#comment133095>
nit: same here.
- Yi Pan (Data Infrastructure)
On May 1, 2015, 6:43 p.m., Guozhang Wang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33761/
> -----------------------------------------------------------
>
> (Updated May 1, 2015, 6:43 p.m.)
>
>
> Review request for samza.
>
>
> Bugs: SAMZA-658
> https://issues.apache.org/jira/browse/SAMZA-658
>
>
> Repository: samza
>
>
> Description
> -------
>
> Add removing logic to the intermediate cached store for kv engine
>
>
> Diffs
> -----
>
> samza-kv/src/main/scala/org/apache/samza/storage/kv/BaseKeyValueStorageEngineFactory.scala b3624e6057ee1a86090f00d2853035b06f63358d
> samza-kv/src/main/scala/org/apache/samza/storage/kv/CachedStore.scala 61bb3f6acb080b653f8b11176538549738255acc
> samza-kv/src/main/scala/org/apache/samza/storage/kv/KeyValueStorageEngine.scala 3a23daf053f0b8dec3a7ec83a51c9c5527078a3b
>
> Diff: https://reviews.apache.org/r/33761/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Guozhang Wang
>
>
Re: Review Request 33761: Fix SAMZA-658
Posted by Guozhang Wang <wa...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33761/
-----------------------------------------------------------
(Updated May 1, 2015, 6:43 p.m.)
Review request for samza.
Bugs: SAMZA-658
https://issues.apache.org/jira/browse/SAMZA-658
Repository: samza
Description
-------
Add removing logic to the intermediate cached store for kv engine
Diffs
-----
samza-kv/src/main/scala/org/apache/samza/storage/kv/BaseKeyValueStorageEngineFactory.scala b3624e6057ee1a86090f00d2853035b06f63358d
samza-kv/src/main/scala/org/apache/samza/storage/kv/CachedStore.scala 61bb3f6acb080b653f8b11176538549738255acc
samza-kv/src/main/scala/org/apache/samza/storage/kv/KeyValueStorageEngine.scala 3a23daf053f0b8dec3a7ec83a51c9c5527078a3b
Diff: https://reviews.apache.org/r/33761/diff/
Testing
-------
Thanks,
Guozhang Wang