You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@samza.apache.org by Chris Riccomini <cr...@apache.org> on 2014/02/24 21:46:45 UTC

Review Request 18437: SAMZA-148

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18437/
-----------------------------------------------------------

Review request for samza.


Bugs: SAMZA-148
    https://issues.apache.org/jira/browse/SAMZA-148


Repository: samza


Description
-------

Change SystemConsumer.register to take offset to start reading from, not last offset read.


Diffs
-----

  samza-api/src/main/java/org/apache/samza/system/SystemConsumer.java cf95996 
  samza-api/src/main/java/org/apache/samza/system/SystemStreamMetadata.java 36c57f6 
  samza-api/src/main/java/org/apache/samza/system/chooser/BaseMessageChooser.java 1c280c9 
  samza-api/src/main/java/org/apache/samza/system/chooser/MessageChooser.java 647b7f7 
  samza-api/src/main/java/org/apache/samza/util/BlockingEnvelopeMap.java ab4a48f 
  samza-api/src/test/java/org/apache/samza/util/TestSinglePartitionWithoutOffsetsSystemAdmin.java 4686009 
  samza-core/src/main/scala/org/apache/samza/system/SystemConsumers.scala d70c91b 
  samza-core/src/main/scala/org/apache/samza/system/chooser/BatchingChooser.scala b265454 
  samza-core/src/main/scala/org/apache/samza/system/chooser/BootstrappingChooser.scala 41dbf87 
  samza-core/src/main/scala/org/apache/samza/system/chooser/DefaultChooser.scala bacf32b 
  samza-core/src/main/scala/org/apache/samza/system/chooser/TieredPriorityChooser.scala 24c6875 
  samza-core/src/test/scala/org/apache/samza/system/TestSystemConsumers.scala c9ae187 
  samza-core/src/test/scala/org/apache/samza/system/chooser/MockMessageChooser.scala d8a2f78 
  samza-core/src/test/scala/org/apache/samza/system/chooser/TestBootstrappingChooser.scala b0c1170 
  samza-core/src/test/scala/org/apache/samza/system/chooser/TestDefaultChooser.scala a4b2ec8 
  samza-kafka/src/main/scala/org/apache/samza/system/kafka/BrokerProxy.scala 34727e9 
  samza-kafka/src/main/scala/org/apache/samza/system/kafka/GetOffset.scala 25cd52c 
  samza-kafka/src/main/scala/org/apache/samza/system/kafka/KafkaSystemAdmin.scala 3861b30 
  samza-kafka/src/main/scala/org/apache/samza/system/kafka/KafkaSystemConsumer.scala ed09ef5 
  samza-kafka/src/main/scala/org/apache/samza/system/kafka/MessageSink.scala 1ab0346 
  samza-kafka/src/test/scala/org/apache/samza/system/kafka/TestBrokerProxy.scala d1df781 
  samza-test/src/main/java/org/apache/samza/system/mock/MockSystemAdmin.java 71ca451 
  samza-test/src/main/java/org/apache/samza/system/mock/MockSystemConsumer.java 7ff3111 

Diff: https://reviews.apache.org/r/18437/diff/


Testing
-------


Thanks,

Chris Riccomini

Re: Review Request 18437: SAMZA-148

Posted by Jakob Homan <jg...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18437/#review35458
-----------------------------------------------------------



samza-core/src/main/scala/org/apache/samza/system/chooser/BootstrappingChooser.scala
<https://reviews.apache.org/r/18437/#comment65983>

    Can you add javadoc?



samza-core/src/main/scala/org/apache/samza/system/chooser/BootstrappingChooser.scala
<https://reviews.apache.org/r/18437/#comment65985>

    throw an exception here?



samza-core/src/main/scala/org/apache/samza/system/chooser/BootstrappingChooser.scala
<https://reviews.apache.org/r/18437/#comment65984>

    This compiles? What's the value of offsetToCheck in this case?
    
    Also, indentation is all messed up.



samza-core/src/main/scala/org/apache/samza/system/chooser/BootstrappingChooser.scala
<https://reviews.apache.org/r/18437/#comment65986>

    These should be private so they don't leak, or promoted to more widespread use, so they do.


- Jakob Homan


On Feb. 24, 2014, 12:46 p.m., Chris Riccomini wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18437/
> -----------------------------------------------------------
> 
> (Updated Feb. 24, 2014, 12:46 p.m.)
> 
> 
> Review request for samza.
> 
> 
> Bugs: SAMZA-148
>     https://issues.apache.org/jira/browse/SAMZA-148
> 
> 
> Repository: samza
> 
> 
> Description
> -------
> 
> Change SystemConsumer.register to take offset to start reading from, not last offset read.
> 
> 
> Diffs
> -----
> 
>   samza-api/src/main/java/org/apache/samza/system/SystemConsumer.java cf95996 
>   samza-api/src/main/java/org/apache/samza/system/SystemStreamMetadata.java 36c57f6 
>   samza-api/src/main/java/org/apache/samza/system/chooser/BaseMessageChooser.java 1c280c9 
>   samza-api/src/main/java/org/apache/samza/system/chooser/MessageChooser.java 647b7f7 
>   samza-api/src/main/java/org/apache/samza/util/BlockingEnvelopeMap.java ab4a48f 
>   samza-api/src/test/java/org/apache/samza/util/TestSinglePartitionWithoutOffsetsSystemAdmin.java 4686009 
>   samza-core/src/main/scala/org/apache/samza/system/SystemConsumers.scala d70c91b 
>   samza-core/src/main/scala/org/apache/samza/system/chooser/BatchingChooser.scala b265454 
>   samza-core/src/main/scala/org/apache/samza/system/chooser/BootstrappingChooser.scala 41dbf87 
>   samza-core/src/main/scala/org/apache/samza/system/chooser/DefaultChooser.scala bacf32b 
>   samza-core/src/main/scala/org/apache/samza/system/chooser/TieredPriorityChooser.scala 24c6875 
>   samza-core/src/test/scala/org/apache/samza/system/TestSystemConsumers.scala c9ae187 
>   samza-core/src/test/scala/org/apache/samza/system/chooser/MockMessageChooser.scala d8a2f78 
>   samza-core/src/test/scala/org/apache/samza/system/chooser/TestBootstrappingChooser.scala b0c1170 
>   samza-core/src/test/scala/org/apache/samza/system/chooser/TestDefaultChooser.scala a4b2ec8 
>   samza-kafka/src/main/scala/org/apache/samza/system/kafka/BrokerProxy.scala 34727e9 
>   samza-kafka/src/main/scala/org/apache/samza/system/kafka/GetOffset.scala 25cd52c 
>   samza-kafka/src/main/scala/org/apache/samza/system/kafka/KafkaSystemAdmin.scala 3861b30 
>   samza-kafka/src/main/scala/org/apache/samza/system/kafka/KafkaSystemConsumer.scala ed09ef5 
>   samza-kafka/src/main/scala/org/apache/samza/system/kafka/MessageSink.scala 1ab0346 
>   samza-kafka/src/test/scala/org/apache/samza/system/kafka/TestBrokerProxy.scala d1df781 
>   samza-test/src/main/java/org/apache/samza/system/mock/MockSystemAdmin.java 71ca451 
>   samza-test/src/main/java/org/apache/samza/system/mock/MockSystemConsumer.java 7ff3111 
> 
> Diff: https://reviews.apache.org/r/18437/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Chris Riccomini
> 
>

Re: Review Request 18437: SAMZA-148

Posted by Jakob Homan <jg...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18437/#review35471
-----------------------------------------------------------

Ship it!


Ship It!

- Jakob Homan


On Feb. 25, 2014, 1:36 p.m., Chris Riccomini wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18437/
> -----------------------------------------------------------
> 
> (Updated Feb. 25, 2014, 1:36 p.m.)
> 
> 
> Review request for samza.
> 
> 
> Bugs: SAMZA-148
>     https://issues.apache.org/jira/browse/SAMZA-148
> 
> 
> Repository: samza
> 
> 
> Description
> -------
> 
> Change SystemConsumer.register to take offset to start reading from, not last offset read.
> 
> 
> Diffs
> -----
> 
>   samza-api/src/main/java/org/apache/samza/system/SystemConsumer.java cf95996 
>   samza-api/src/main/java/org/apache/samza/system/SystemStreamMetadata.java 36c57f6 
>   samza-api/src/main/java/org/apache/samza/system/chooser/BaseMessageChooser.java 1c280c9 
>   samza-api/src/main/java/org/apache/samza/system/chooser/MessageChooser.java 647b7f7 
>   samza-api/src/main/java/org/apache/samza/util/BlockingEnvelopeMap.java ab4a48f 
>   samza-api/src/test/java/org/apache/samza/util/TestSinglePartitionWithoutOffsetsSystemAdmin.java 4686009 
>   samza-core/src/main/scala/org/apache/samza/system/SystemConsumers.scala d70c91b 
>   samza-core/src/main/scala/org/apache/samza/system/chooser/BatchingChooser.scala b265454 
>   samza-core/src/main/scala/org/apache/samza/system/chooser/BootstrappingChooser.scala 41dbf87 
>   samza-core/src/main/scala/org/apache/samza/system/chooser/DefaultChooser.scala bacf32b 
>   samza-core/src/main/scala/org/apache/samza/system/chooser/TieredPriorityChooser.scala 24c6875 
>   samza-core/src/test/scala/org/apache/samza/system/TestSystemConsumers.scala c9ae187 
>   samza-core/src/test/scala/org/apache/samza/system/chooser/MockMessageChooser.scala d8a2f78 
>   samza-core/src/test/scala/org/apache/samza/system/chooser/TestBootstrappingChooser.scala b0c1170 
>   samza-core/src/test/scala/org/apache/samza/system/chooser/TestDefaultChooser.scala a4b2ec8 
>   samza-kafka/src/main/scala/org/apache/samza/system/kafka/BrokerProxy.scala 34727e9 
>   samza-kafka/src/main/scala/org/apache/samza/system/kafka/GetOffset.scala 25cd52c 
>   samza-kafka/src/main/scala/org/apache/samza/system/kafka/KafkaSystemAdmin.scala 3861b30 
>   samza-kafka/src/main/scala/org/apache/samza/system/kafka/KafkaSystemConsumer.scala ed09ef5 
>   samza-kafka/src/main/scala/org/apache/samza/system/kafka/MessageSink.scala 1ab0346 
>   samza-kafka/src/test/scala/org/apache/samza/system/kafka/TestBrokerProxy.scala d1df781 
>   samza-test/src/main/java/org/apache/samza/system/mock/MockSystemAdmin.java 71ca451 
>   samza-test/src/main/java/org/apache/samza/system/mock/MockSystemConsumer.java 7ff3111 
> 
> Diff: https://reviews.apache.org/r/18437/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Chris Riccomini
> 
>

Re: Review Request 18437: SAMZA-148

Posted by Chris Riccomini <cr...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18437/
-----------------------------------------------------------

(Updated Feb. 25, 2014, 9:36 p.m.)


Review request for samza.


Changes
-------

Updating changes based off of Jakob's feedback.


Bugs: SAMZA-148
    https://issues.apache.org/jira/browse/SAMZA-148


Repository: samza


Description
-------

Change SystemConsumer.register to take offset to start reading from, not last offset read.


Diffs (updated)
-----

  samza-api/src/main/java/org/apache/samza/system/SystemConsumer.java cf95996 
  samza-api/src/main/java/org/apache/samza/system/SystemStreamMetadata.java 36c57f6 
  samza-api/src/main/java/org/apache/samza/system/chooser/BaseMessageChooser.java 1c280c9 
  samza-api/src/main/java/org/apache/samza/system/chooser/MessageChooser.java 647b7f7 
  samza-api/src/main/java/org/apache/samza/util/BlockingEnvelopeMap.java ab4a48f 
  samza-api/src/test/java/org/apache/samza/util/TestSinglePartitionWithoutOffsetsSystemAdmin.java 4686009 
  samza-core/src/main/scala/org/apache/samza/system/SystemConsumers.scala d70c91b 
  samza-core/src/main/scala/org/apache/samza/system/chooser/BatchingChooser.scala b265454 
  samza-core/src/main/scala/org/apache/samza/system/chooser/BootstrappingChooser.scala 41dbf87 
  samza-core/src/main/scala/org/apache/samza/system/chooser/DefaultChooser.scala bacf32b 
  samza-core/src/main/scala/org/apache/samza/system/chooser/TieredPriorityChooser.scala 24c6875 
  samza-core/src/test/scala/org/apache/samza/system/TestSystemConsumers.scala c9ae187 
  samza-core/src/test/scala/org/apache/samza/system/chooser/MockMessageChooser.scala d8a2f78 
  samza-core/src/test/scala/org/apache/samza/system/chooser/TestBootstrappingChooser.scala b0c1170 
  samza-core/src/test/scala/org/apache/samza/system/chooser/TestDefaultChooser.scala a4b2ec8 
  samza-kafka/src/main/scala/org/apache/samza/system/kafka/BrokerProxy.scala 34727e9 
  samza-kafka/src/main/scala/org/apache/samza/system/kafka/GetOffset.scala 25cd52c 
  samza-kafka/src/main/scala/org/apache/samza/system/kafka/KafkaSystemAdmin.scala 3861b30 
  samza-kafka/src/main/scala/org/apache/samza/system/kafka/KafkaSystemConsumer.scala ed09ef5 
  samza-kafka/src/main/scala/org/apache/samza/system/kafka/MessageSink.scala 1ab0346 
  samza-kafka/src/test/scala/org/apache/samza/system/kafka/TestBrokerProxy.scala d1df781 
  samza-test/src/main/java/org/apache/samza/system/mock/MockSystemAdmin.java 71ca451 
  samza-test/src/main/java/org/apache/samza/system/mock/MockSystemConsumer.java 7ff3111 

Diff: https://reviews.apache.org/r/18437/diff/


Testing
-------


Thanks,

Chris Riccomini

Re: Review Request 18437: SAMZA-148

Posted by Chris Riccomini <cr...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18437/#review35328
-----------------------------------------------------------



samza-core/src/main/scala/org/apache/samza/system/chooser/BootstrappingChooser.scala
<https://reviews.apache.org/r/18437/#comment65788>

    Don't actually think that we use this anywhere.



samza-core/src/main/scala/org/apache/samza/system/chooser/BootstrappingChooser.scala
<https://reviews.apache.org/r/18437/#comment65789>

    Should be Upcoming instead of Future.



samza-core/src/main/scala/org/apache/samza/system/chooser/BootstrappingChooser.scala
<https://reviews.apache.org/r/18437/#comment65787>

    Should throw exception here.



samza-kafka/src/main/scala/org/apache/samza/system/kafka/GetOffset.scala
<https://reviews.apache.org/r/18437/#comment65792>

    Don't need this if we use toss instead.



samza-kafka/src/main/scala/org/apache/samza/system/kafka/GetOffset.scala
<https://reviews.apache.org/r/18437/#comment65791>

    Use toss here.


- Chris Riccomini


On Feb. 24, 2014, 8:46 p.m., Chris Riccomini wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18437/
> -----------------------------------------------------------
> 
> (Updated Feb. 24, 2014, 8:46 p.m.)
> 
> 
> Review request for samza.
> 
> 
> Bugs: SAMZA-148
>     https://issues.apache.org/jira/browse/SAMZA-148
> 
> 
> Repository: samza
> 
> 
> Description
> -------
> 
> Change SystemConsumer.register to take offset to start reading from, not last offset read.
> 
> 
> Diffs
> -----
> 
>   samza-api/src/main/java/org/apache/samza/system/SystemConsumer.java cf95996 
>   samza-api/src/main/java/org/apache/samza/system/SystemStreamMetadata.java 36c57f6 
>   samza-api/src/main/java/org/apache/samza/system/chooser/BaseMessageChooser.java 1c280c9 
>   samza-api/src/main/java/org/apache/samza/system/chooser/MessageChooser.java 647b7f7 
>   samza-api/src/main/java/org/apache/samza/util/BlockingEnvelopeMap.java ab4a48f 
>   samza-api/src/test/java/org/apache/samza/util/TestSinglePartitionWithoutOffsetsSystemAdmin.java 4686009 
>   samza-core/src/main/scala/org/apache/samza/system/SystemConsumers.scala d70c91b 
>   samza-core/src/main/scala/org/apache/samza/system/chooser/BatchingChooser.scala b265454 
>   samza-core/src/main/scala/org/apache/samza/system/chooser/BootstrappingChooser.scala 41dbf87 
>   samza-core/src/main/scala/org/apache/samza/system/chooser/DefaultChooser.scala bacf32b 
>   samza-core/src/main/scala/org/apache/samza/system/chooser/TieredPriorityChooser.scala 24c6875 
>   samza-core/src/test/scala/org/apache/samza/system/TestSystemConsumers.scala c9ae187 
>   samza-core/src/test/scala/org/apache/samza/system/chooser/MockMessageChooser.scala d8a2f78 
>   samza-core/src/test/scala/org/apache/samza/system/chooser/TestBootstrappingChooser.scala b0c1170 
>   samza-core/src/test/scala/org/apache/samza/system/chooser/TestDefaultChooser.scala a4b2ec8 
>   samza-kafka/src/main/scala/org/apache/samza/system/kafka/BrokerProxy.scala 34727e9 
>   samza-kafka/src/main/scala/org/apache/samza/system/kafka/GetOffset.scala 25cd52c 
>   samza-kafka/src/main/scala/org/apache/samza/system/kafka/KafkaSystemAdmin.scala 3861b30 
>   samza-kafka/src/main/scala/org/apache/samza/system/kafka/KafkaSystemConsumer.scala ed09ef5 
>   samza-kafka/src/main/scala/org/apache/samza/system/kafka/MessageSink.scala 1ab0346 
>   samza-kafka/src/test/scala/org/apache/samza/system/kafka/TestBrokerProxy.scala d1df781 
>   samza-test/src/main/java/org/apache/samza/system/mock/MockSystemAdmin.java 71ca451 
>   samza-test/src/main/java/org/apache/samza/system/mock/MockSystemConsumer.java 7ff3111 
> 
> Diff: https://reviews.apache.org/r/18437/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Chris Riccomini
> 
>