You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@samza.apache.org by Robert Žuljevi? <r....@levi9.com> on 2015/06/04 15:50:10 UTC
Review Request 35067: SAMZA-546: Add config option
max.in.flight.requests.per.connection
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35067/
-----------------------------------------------------------
Review request for samza.
Repository: samza
Description
-------
Added config max.in.flight.requests.per.connection and retries options for Kafka producers.
Diffs
-----
samza-kafka/src/main/scala/org/apache/samza/config/KafkaConfig.scala a1de88781a8ff2a3fe50389ae7a59062289404fc
Diff: https://reviews.apache.org/r/35067/diff/
Testing
-------
Thanks,
Robert Žuljevi?
Re: Review Request 35067: SAMZA-546: Add config option
max.in.flight.requests.per.connection
Posted by Navina Ramesh <nr...@linkedin.com>.
> On June 8, 2015, 2:29 a.m., Navina Ramesh wrote:
> > samza-kafka/src/main/scala/org/apache/samza/config/KafkaConfig.scala, line 182
> > <https://reviews.apache.org/r/35067/diff/1/?file=979072#file979072line182>
> >
> > This warning is valid only when the configured value for max.in.flight.requests.per.connection > 1. Can you check the value and make the comment more elaborate?
>
> Robert Zuljevic wrote:
> Hi. In order to do that I would need to cast the string to integer which can throw an NumberFormatException. Is this ok?
Yeah. I understand your concern here. I think config validation is not yet standardized in the codebase. I think this should be fixed when we address SAMZA-471
- Navina
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35067/#review86980
-----------------------------------------------------------
On June 15, 2015, 1:42 p.m., Robert Zuljevic wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35067/
> -----------------------------------------------------------
>
> (Updated June 15, 2015, 1:42 p.m.)
>
>
> Review request for samza.
>
>
> Repository: samza
>
>
> Description
> -------
>
> Fixed test
>
>
> Diffs
> -----
>
> samza-kafka/src/main/scala/org/apache/samza/config/KafkaConfig.scala a1de88781a8ff2a3fe50389ae7a59062289404fc
> samza-test/src/test/scala/org/apache/samza/test/integration/TestStatefulTask.scala 82006960adc45a92a79e28f078ebd5eb01d3205e
>
> Diff: https://reviews.apache.org/r/35067/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Robert Zuljevic
>
>
Re: Review Request 35067: SAMZA-546: Add config option
max.in.flight.requests.per.connection
Posted by Robert Zuljevic <r....@levi9.com>.
> On June 8, 2015, 2:29 a.m., Navina Ramesh wrote:
> > samza-kafka/src/main/scala/org/apache/samza/config/KafkaConfig.scala, line 182
> > <https://reviews.apache.org/r/35067/diff/1/?file=979072#file979072line182>
> >
> > This warning is valid only when the configured value for max.in.flight.requests.per.connection > 1. Can you check the value and make the comment more elaborate?
Hi. In order to do that I would need to cast the string to integer which can throw an NumberFormatException. Is this ok?
- Robert
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35067/#review86980
-----------------------------------------------------------
On June 4, 2015, 1:50 p.m., Robert Zuljevic wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35067/
> -----------------------------------------------------------
>
> (Updated June 4, 2015, 1:50 p.m.)
>
>
> Review request for samza.
>
>
> Repository: samza
>
>
> Description
> -------
>
> Added config max.in.flight.requests.per.connection and retries options for Kafka producers.
>
>
> Diffs
> -----
>
> samza-kafka/src/main/scala/org/apache/samza/config/KafkaConfig.scala a1de88781a8ff2a3fe50389ae7a59062289404fc
>
> Diff: https://reviews.apache.org/r/35067/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Robert Zuljevic
>
>
Re: Review Request 35067: SAMZA-546: Add config option
max.in.flight.requests.per.connection
Posted by Navina Ramesh <nr...@linkedin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35067/#review86980
-----------------------------------------------------------
Apart from the comments, lgtm!
samza-kafka/src/main/scala/org/apache/samza/config/KafkaConfig.scala
<https://reviews.apache.org/r/35067/#comment139190>
This warning is valid only when the configured value for max.in.flight.requests.per.connection > 1. Can you check the value and make the comment more elaborate?
samza-kafka/src/main/scala/org/apache/samza/config/KafkaConfig.scala
<https://reviews.apache.org/r/35067/#comment139191>
We need a log message when this value is overriden (to < INT.MAX) and that Samza cannot handle producer failures. Hence, the onus of retries is left up to the kafka producer.
- Navina Ramesh
On June 4, 2015, 1:50 p.m., Robert Zuljevic wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35067/
> -----------------------------------------------------------
>
> (Updated June 4, 2015, 1:50 p.m.)
>
>
> Review request for samza.
>
>
> Repository: samza
>
>
> Description
> -------
>
> Added config max.in.flight.requests.per.connection and retries options for Kafka producers.
>
>
> Diffs
> -----
>
> samza-kafka/src/main/scala/org/apache/samza/config/KafkaConfig.scala a1de88781a8ff2a3fe50389ae7a59062289404fc
>
> Diff: https://reviews.apache.org/r/35067/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Robert Zuljevic
>
>
Re: Review Request 35067: SAMZA-546: Add config option
max.in.flight.requests.per.connection
Posted by Robert Zuljevic <r....@levi9.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35067/
-----------------------------------------------------------
(Updated June 16, 2015, 7:47 a.m.)
Review request for samza.
Repository: samza
Description (updated)
-------
Added tests for new Kafka config properties
Diffs (updated)
-----
samza-kafka/src/main/scala/org/apache/samza/config/KafkaConfig.scala a1de88781a8ff2a3fe50389ae7a59062289404fc
samza-kafka/src/test/scala/org/apache/samza/config/TestKafkaConfig.scala 95f76b61f70058bb567fe4eff3a3bb2298a0e197
samza-test/src/test/scala/org/apache/samza/test/integration/TestStatefulTask.scala 82006960adc45a92a79e28f078ebd5eb01d3205e
Diff: https://reviews.apache.org/r/35067/diff/
Testing
-------
Thanks,
Robert Zuljevic
Re: Review Request 35067: SAMZA-546: Add config option
max.in.flight.requests.per.connection
Posted by Navina Ramesh <nr...@linkedin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35067/#review87952
-----------------------------------------------------------
Ship it!
Ship It!
- Navina Ramesh
On June 15, 2015, 1:42 p.m., Robert Zuljevic wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35067/
> -----------------------------------------------------------
>
> (Updated June 15, 2015, 1:42 p.m.)
>
>
> Review request for samza.
>
>
> Repository: samza
>
>
> Description
> -------
>
> Fixed test
>
>
> Diffs
> -----
>
> samza-kafka/src/main/scala/org/apache/samza/config/KafkaConfig.scala a1de88781a8ff2a3fe50389ae7a59062289404fc
> samza-test/src/test/scala/org/apache/samza/test/integration/TestStatefulTask.scala 82006960adc45a92a79e28f078ebd5eb01d3205e
>
> Diff: https://reviews.apache.org/r/35067/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Robert Zuljevic
>
>
Re: Review Request 35067: SAMZA-546: Add config option
max.in.flight.requests.per.connection
Posted by Navina Ramesh <nr...@linkedin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35067/#review87963
-----------------------------------------------------------
@Robert:
Can you please add a unit test in TestKafkaConfig to verify that these properties get overridden correctly ? Sorry about this afterthought request.
- Navina Ramesh
On June 15, 2015, 1:42 p.m., Robert Zuljevic wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35067/
> -----------------------------------------------------------
>
> (Updated June 15, 2015, 1:42 p.m.)
>
>
> Review request for samza.
>
>
> Repository: samza
>
>
> Description
> -------
>
> Fixed test
>
>
> Diffs
> -----
>
> samza-kafka/src/main/scala/org/apache/samza/config/KafkaConfig.scala a1de88781a8ff2a3fe50389ae7a59062289404fc
> samza-test/src/test/scala/org/apache/samza/test/integration/TestStatefulTask.scala 82006960adc45a92a79e28f078ebd5eb01d3205e
>
> Diff: https://reviews.apache.org/r/35067/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Robert Zuljevic
>
>
Re: Review Request 35067: SAMZA-546: Add config option
max.in.flight.requests.per.connection
Posted by Robert Zuljevic <r....@levi9.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35067/
-----------------------------------------------------------
(Updated June 15, 2015, 1:42 p.m.)
Review request for samza.
Repository: samza
Description (updated)
-------
Fixed test
Diffs (updated)
-----
samza-kafka/src/main/scala/org/apache/samza/config/KafkaConfig.scala a1de88781a8ff2a3fe50389ae7a59062289404fc
samza-test/src/test/scala/org/apache/samza/test/integration/TestStatefulTask.scala 82006960adc45a92a79e28f078ebd5eb01d3205e
Diff: https://reviews.apache.org/r/35067/diff/
Testing
-------
Thanks,
Robert Zuljevic