You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@samza.apache.org by József Márton Jung <j....@levi9.com> on 2015/07/22 16:03:14 UTC
Review Request 36692: SAMZA-717 Expose the TaskNameGrouper API
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36692/
-----------------------------------------------------------
Review request for samza.
Repository: samza
Description
-------
TaskNameGrouper is exposed (and renamed to TaskGrouper) as an API that users can use to implement how they want to assign the TaskModels to the ContainerModels.
Diffs
-----
docs/learn/documentation/versioned/jobs/configuration-table.html 6aa7b91
samza-api/src/main/java/org/apache/samza/container/grouper/task/TaskGrouper.java PRE-CREATION
samza-api/src/main/java/org/apache/samza/container/grouper/task/TaskGrouperFactory.java PRE-CREATION
samza-api/src/main/java/org/apache/samza/job/model/ContainerModel.java PRE-CREATION
samza-api/src/main/java/org/apache/samza/job/model/TaskModel.java PRE-CREATION
samza-core/src/main/java/org/apache/samza/job/model/ContainerModel.java 98a34bc
samza-core/src/main/java/org/apache/samza/job/model/TaskModel.java c26690a
samza-core/src/main/scala/org/apache/samza/config/TaskConfig.scala 0b3a235
samza-core/src/main/scala/org/apache/samza/container/grouper/task/GroupByContainerCount.scala be36125
samza-core/src/main/scala/org/apache/samza/container/grouper/task/GroupByContainerCountFactory.scala PRE-CREATION
samza-core/src/main/scala/org/apache/samza/container/grouper/task/TaskNameGrouper.scala 62e94ea
samza-core/src/main/scala/org/apache/samza/coordinator/JobCoordinator.scala f621611
samza-core/src/test/java/org/apache/samza/storage/TestStorageRecovery.java b8ae592
samza-core/src/test/scala/org/apache/samza/checkpoint/TestCheckpointTool.scala 0ba932c
samza-core/src/test/scala/org/apache/samza/container/grouper/task/TestGroupByContainerCount.scala 2c7cb28
samza-core/src/test/scala/org/apache/samza/coordinator/TestJobCoordinator.scala d9ae187
Diff: https://reviews.apache.org/r/36692/diff/
Testing
-------
Tests are updated and they cover the modifications. Also tested with hello-samza.
Thanks,
József Márton Jung
Re: Review Request 36692: SAMZA-717 Expose the TaskNameGrouper API
Posted by "Gustavo Anatoly F. V. Solís" <gu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36692/#review92594
-----------------------------------------------------------
samza-api/src/main/java/org/apache/samza/job/model/ContainerModel.java (line 32)
<https://reviews.apache.org/r/36692/#comment146791>
Remove trailing spaces
- Gustavo Anatoly F. V. Solís
On Julho 22, 2015, 2:03 p.m., József Márton Jung wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36692/
> -----------------------------------------------------------
>
> (Updated Julho 22, 2015, 2:03 p.m.)
>
>
> Review request for samza.
>
>
> Repository: samza
>
>
> Description
> -------
>
> TaskNameGrouper is exposed (and renamed to TaskGrouper) as an API that users can use to implement how they want to assign the TaskModels to the ContainerModels.
>
>
> Diffs
> -----
>
> docs/learn/documentation/versioned/jobs/configuration-table.html 6aa7b91
> samza-api/src/main/java/org/apache/samza/container/grouper/task/TaskGrouper.java PRE-CREATION
> samza-api/src/main/java/org/apache/samza/container/grouper/task/TaskGrouperFactory.java PRE-CREATION
> samza-api/src/main/java/org/apache/samza/job/model/ContainerModel.java PRE-CREATION
> samza-api/src/main/java/org/apache/samza/job/model/TaskModel.java PRE-CREATION
> samza-core/src/main/java/org/apache/samza/job/model/ContainerModel.java 98a34bc
> samza-core/src/main/java/org/apache/samza/job/model/TaskModel.java c26690a
> samza-core/src/main/scala/org/apache/samza/config/TaskConfig.scala 0b3a235
> samza-core/src/main/scala/org/apache/samza/container/grouper/task/GroupByContainerCount.scala be36125
> samza-core/src/main/scala/org/apache/samza/container/grouper/task/GroupByContainerCountFactory.scala PRE-CREATION
> samza-core/src/main/scala/org/apache/samza/container/grouper/task/TaskNameGrouper.scala 62e94ea
> samza-core/src/main/scala/org/apache/samza/coordinator/JobCoordinator.scala f621611
> samza-core/src/test/java/org/apache/samza/storage/TestStorageRecovery.java b8ae592
> samza-core/src/test/scala/org/apache/samza/checkpoint/TestCheckpointTool.scala 0ba932c
> samza-core/src/test/scala/org/apache/samza/container/grouper/task/TestGroupByContainerCount.scala 2c7cb28
> samza-core/src/test/scala/org/apache/samza/coordinator/TestJobCoordinator.scala d9ae187
>
> Diff: https://reviews.apache.org/r/36692/diff/
>
>
> Testing
> -------
>
> Tests are updated and they cover the modifications. Also tested with hello-samza.
>
>
> Thanks,
>
> József Márton Jung
>
>
Re: Review Request 36692: SAMZA-717 Expose the TaskNameGrouper API
Posted by "Yi Pan (Data Infrastructure)" <yi...@linkedin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36692/#review93097
-----------------------------------------------------------
Ship it!
The code LGTM. I just realized that when we expose this via configuration, we will need to make sure that we will phase out this configure variables when we refactor the code into the PartitionManager later. It is OK if we can just map this configure to an implementation of the new partition manager class.
- Yi Pan (Data Infrastructure)
On July 23, 2015, 11:23 a.m., József Márton Jung wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36692/
> -----------------------------------------------------------
>
> (Updated July 23, 2015, 11:23 a.m.)
>
>
> Review request for samza.
>
>
> Repository: samza
>
>
> Description
> -------
>
> TaskNameGrouper is exposed (and renamed to TaskGrouper) as an API that users can use to implement how they want to assign the TaskModels to the ContainerModels.
>
>
> Diffs
> -----
>
> docs/learn/documentation/versioned/jobs/configuration-table.html 6aa7b91
> samza-core/src/main/java/org/apache/samza/container/grouper/task/TaskNameGrouper.java PRE-CREATION
> samza-core/src/main/java/org/apache/samza/container/grouper/task/TaskNameGrouperFactory.java PRE-CREATION
> samza-core/src/main/java/org/apache/samza/job/model/ContainerModel.java 98a34bc
> samza-core/src/main/java/org/apache/samza/job/model/TaskModel.java c26690a
> samza-core/src/main/scala/org/apache/samza/config/TaskConfig.scala 0b3a235
> samza-core/src/main/scala/org/apache/samza/container/grouper/task/GroupByContainerCount.scala be36125
> samza-core/src/main/scala/org/apache/samza/container/grouper/task/GroupByContainerCountFactory.scala PRE-CREATION
> samza-core/src/main/scala/org/apache/samza/container/grouper/task/TaskNameGrouper.scala 62e94ea
> samza-core/src/main/scala/org/apache/samza/coordinator/JobCoordinator.scala f621611
> samza-core/src/test/java/org/apache/samza/storage/TestStorageRecovery.java b8ae592
> samza-core/src/test/scala/org/apache/samza/checkpoint/TestCheckpointTool.scala 0ba932c
> samza-core/src/test/scala/org/apache/samza/container/grouper/task/TestGroupByContainerCount.scala 2c7cb28
> samza-core/src/test/scala/org/apache/samza/coordinator/TestJobCoordinator.scala d9ae187
>
> Diff: https://reviews.apache.org/r/36692/diff/
>
>
> Testing
> -------
>
> Tests are updated and they cover the modifications. Also tested with hello-samza.
>
>
> Thanks,
>
> József Márton Jung
>
>
Re: Review Request 36692: SAMZA-717 Expose the TaskNameGrouper API
Posted by József Márton Jung <j....@levi9.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36692/
-----------------------------------------------------------
(Updated July 23, 2015, 11:23 a.m.)
Review request for samza.
Repository: samza
Description
-------
TaskNameGrouper is exposed (and renamed to TaskGrouper) as an API that users can use to implement how they want to assign the TaskModels to the ContainerModels.
Diffs (updated)
-----
docs/learn/documentation/versioned/jobs/configuration-table.html 6aa7b91
samza-core/src/main/java/org/apache/samza/container/grouper/task/TaskNameGrouper.java PRE-CREATION
samza-core/src/main/java/org/apache/samza/container/grouper/task/TaskNameGrouperFactory.java PRE-CREATION
samza-core/src/main/java/org/apache/samza/job/model/ContainerModel.java 98a34bc
samza-core/src/main/java/org/apache/samza/job/model/TaskModel.java c26690a
samza-core/src/main/scala/org/apache/samza/config/TaskConfig.scala 0b3a235
samza-core/src/main/scala/org/apache/samza/container/grouper/task/GroupByContainerCount.scala be36125
samza-core/src/main/scala/org/apache/samza/container/grouper/task/GroupByContainerCountFactory.scala PRE-CREATION
samza-core/src/main/scala/org/apache/samza/container/grouper/task/TaskNameGrouper.scala 62e94ea
samza-core/src/main/scala/org/apache/samza/coordinator/JobCoordinator.scala f621611
samza-core/src/test/java/org/apache/samza/storage/TestStorageRecovery.java b8ae592
samza-core/src/test/scala/org/apache/samza/checkpoint/TestCheckpointTool.scala 0ba932c
samza-core/src/test/scala/org/apache/samza/container/grouper/task/TestGroupByContainerCount.scala 2c7cb28
samza-core/src/test/scala/org/apache/samza/coordinator/TestJobCoordinator.scala d9ae187
Diff: https://reviews.apache.org/r/36692/diff/
Testing
-------
Tests are updated and they cover the modifications. Also tested with hello-samza.
Thanks,
József Márton Jung
Re: Review Request 36692: SAMZA-717 Expose the TaskNameGrouper API
Posted by József Márton Jung <j....@levi9.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36692/
-----------------------------------------------------------
(Updated July 23, 2015, 11:16 a.m.)
Review request for samza.
Changes
-------
Corrections based on review and comments on the ticket.
Repository: samza
Description
-------
TaskNameGrouper is exposed (and renamed to TaskGrouper) as an API that users can use to implement how they want to assign the TaskModels to the ContainerModels.
Diffs (updated)
-----
docs/learn/documentation/versioned/jobs/configuration-table.html 6aa7b91
samza-core/src/main/java/org/apache/samza/container/grouper/task/TaskNameGrouper.java PRE-CREATION
samza-core/src/main/java/org/apache/samza/container/grouper/task/TaskNameGrouperFactory.java PRE-CREATION
samza-core/src/main/java/org/apache/samza/job/model/ContainerModel.java 98a34bc
samza-core/src/main/java/org/apache/samza/job/model/TaskModel.java c26690a
samza-core/src/main/scala/org/apache/samza/config/TaskConfig.scala 0b3a235
samza-core/src/main/scala/org/apache/samza/container/grouper/task/GroupByContainerCount.scala be36125
samza-core/src/main/scala/org/apache/samza/container/grouper/task/GroupByContainerCountFactory.scala PRE-CREATION
samza-core/src/main/scala/org/apache/samza/container/grouper/task/TaskNameGrouper.scala 62e94ea
samza-core/src/main/scala/org/apache/samza/coordinator/JobCoordinator.scala f621611
samza-core/src/test/java/org/apache/samza/storage/TestStorageRecovery.java b8ae592
samza-core/src/test/scala/org/apache/samza/checkpoint/TestCheckpointTool.scala 0ba932c
samza-core/src/test/scala/org/apache/samza/container/grouper/task/TestGroupByContainerCount.scala 2c7cb28
samza-core/src/test/scala/org/apache/samza/coordinator/TestJobCoordinator.scala d9ae187
Diff: https://reviews.apache.org/r/36692/diff/
Testing
-------
Tests are updated and they cover the modifications. Also tested with hello-samza.
Thanks,
József Márton Jung