You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Guozhang Wang <gu...@linkedin.com> on 2014/04/08 23:30:06 UTC
Review Request 20131: Fix KAFKA-1353: incorporate Jun's comment round two
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20131/
-----------------------------------------------------------
Review request for kafka.
Bugs: KAFKA-1353
https://issues.apache.org/jira/browse/KAFKA-1353
Repository: kafka
Description
-------
Remove unused imports
KAFKA-1353.v3
KAFKA-1353.v2
KAFKA-1353.v1
Diffs
-----
core/src/main/scala/kafka/network/RequestChannel.scala a6ec970d72fc8adc56bc6e200d1ff5bbf0d2b748
core/src/main/scala/kafka/network/SocketServer.scala dcfca3f4156957b0006ba9bfdeca848c115825a8
core/src/main/scala/kafka/server/KafkaRequestHandler.scala 871212bf9110ca95ca7d2109be070f69d0f90247
Diff: https://reviews.apache.org/r/20131/diff/
Testing
-------
Thanks,
Guozhang Wang
Re: Review Request 20131: Fix KAFKA-1353: incorporate Jun's comment round two
Posted by Jun Rao <ju...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20131/#review39831
-----------------------------------------------------------
core/src/main/scala/kafka/network/SocketServer.scala
<https://reviews.apache.org/r/20131/#comment72514>
How about using the following comment?
We use a single meter for aggregate idle percentage for the thread pool. Since meter is calculated as total_recorded_value / time_window and time_window is independent of the number of threads, each recorded idle time should be discounted by # threads.
core/src/main/scala/kafka/server/KafkaRequestHandler.scala
<https://reviews.apache.org/r/20131/#comment72506>
Perhaps use totalHandlerThreads?
- Jun Rao
On April 8, 2014, 9:56 p.m., Guozhang Wang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20131/
> -----------------------------------------------------------
>
> (Updated April 8, 2014, 9:56 p.m.)
>
>
> Review request for kafka.
>
>
> Bugs: KAFKA-1353
> https://issues.apache.org/jira/browse/KAFKA-1353
>
>
> Repository: kafka
>
>
> Description
> -------
>
> Remove unused imports
>
>
> KAFKA-1353.v3
>
>
> KAFKA-1353.v2
>
>
> KAFKA-1353.v1
>
>
> Diffs
> -----
>
> core/src/main/scala/kafka/network/RequestChannel.scala a6ec970d72fc8adc56bc6e200d1ff5bbf0d2b748
> core/src/main/scala/kafka/network/SocketServer.scala dcfca3f4156957b0006ba9bfdeca848c115825a8
> core/src/main/scala/kafka/server/KafkaRequestHandler.scala 871212bf9110ca95ca7d2109be070f69d0f90247
>
> Diff: https://reviews.apache.org/r/20131/diff/
>
>
> Testing
> -------
>
> Local test with 9 new producer performance and one handler thread: in 3 minutes, the io threads idle time drops to 10%, but after that remains vibrating, the network threads keeps dropping but much slower, only to about 85% in 3 minutes.
>
> Adding to 15 producers, the io thread starts to drop again, until close to 2% where it starts vibrating again, whereas the network threads drops to about 80%.
>
>
> Thanks,
>
> Guozhang Wang
>
>
Re: Review Request 20131: Fix KAFKA-1353: minor changes
Posted by Guozhang Wang <gu...@linkedin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20131/
-----------------------------------------------------------
(Updated April 9, 2014, 5:13 p.m.)
Review request for kafka.
Summary (updated)
-----------------
Fix KAFKA-1353: minor changes
Bugs: KAFKA-1353
https://issues.apache.org/jira/browse/KAFKA-1353
Repository: kafka
Description (updated)
-------
Revert num.io.threads to 8 by default
KAFKA-1353.v4
Remove unused imports
KAFKA-1353.v3
KAFKA-1353.v2
KAFKA-1353.v1
Diffs (updated)
-----
core/src/main/scala/kafka/network/RequestChannel.scala a6ec970d72fc8adc56bc6e200d1ff5bbf0d2b748
core/src/main/scala/kafka/network/SocketServer.scala dcfca3f4156957b0006ba9bfdeca848c115825a8
core/src/main/scala/kafka/server/KafkaRequestHandler.scala 871212bf9110ca95ca7d2109be070f69d0f90247
Diff: https://reviews.apache.org/r/20131/diff/
Testing
-------
Local test with 9 new producer performance and one handler thread: in 3 minutes, the io threads idle time drops to 10%, but after that remains vibrating, the network threads keeps dropping but much slower, only to about 85% in 3 minutes.
Adding to 15 producers, the io thread starts to drop again, until close to 2% where it starts vibrating again, whereas the network threads drops to about 80%.
Thanks,
Guozhang Wang
Re: Review Request 20131: Fix KAFKA-1353: incorporate Jun's comment round
three
Posted by Jun Rao <ju...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20131/#review39841
-----------------------------------------------------------
core/src/main/scala/kafka/server/KafkaConfig.scala
<https://reviews.apache.org/r/20131/#comment72554>
This should be reverted.
- Jun Rao
On April 8, 2014, 11:31 p.m., Guozhang Wang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20131/
> -----------------------------------------------------------
>
> (Updated April 8, 2014, 11:31 p.m.)
>
>
> Review request for kafka.
>
>
> Bugs: KAFKA-1353
> https://issues.apache.org/jira/browse/KAFKA-1353
>
>
> Repository: kafka
>
>
> Description
> -------
>
> KAFKA-1353.v4
>
>
> Remove unused imports
>
>
> KAFKA-1353.v3
>
>
> KAFKA-1353.v2
>
>
> KAFKA-1353.v1
>
>
> Diffs
> -----
>
> core/src/main/scala/kafka/network/RequestChannel.scala a6ec970d72fc8adc56bc6e200d1ff5bbf0d2b748
> core/src/main/scala/kafka/network/SocketServer.scala dcfca3f4156957b0006ba9bfdeca848c115825a8
> core/src/main/scala/kafka/server/KafkaConfig.scala b0506d4881176cd95914847c4275e428a8d2ca10
> core/src/main/scala/kafka/server/KafkaRequestHandler.scala 871212bf9110ca95ca7d2109be070f69d0f90247
>
> Diff: https://reviews.apache.org/r/20131/diff/
>
>
> Testing
> -------
>
> Local test with 9 new producer performance and one handler thread: in 3 minutes, the io threads idle time drops to 10%, but after that remains vibrating, the network threads keeps dropping but much slower, only to about 85% in 3 minutes.
>
> Adding to 15 producers, the io thread starts to drop again, until close to 2% where it starts vibrating again, whereas the network threads drops to about 80%.
>
>
> Thanks,
>
> Guozhang Wang
>
>
Re: Review Request 20131: Fix KAFKA-1353: incorporate Jun's comment round
three
Posted by Guozhang Wang <gu...@linkedin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20131/
-----------------------------------------------------------
(Updated April 8, 2014, 11:31 p.m.)
Review request for kafka.
Summary (updated)
-----------------
Fix KAFKA-1353: incorporate Jun's comment round three
Bugs: KAFKA-1353
https://issues.apache.org/jira/browse/KAFKA-1353
Repository: kafka
Description (updated)
-------
KAFKA-1353.v4
Remove unused imports
KAFKA-1353.v3
KAFKA-1353.v2
KAFKA-1353.v1
Diffs (updated)
-----
core/src/main/scala/kafka/network/RequestChannel.scala a6ec970d72fc8adc56bc6e200d1ff5bbf0d2b748
core/src/main/scala/kafka/network/SocketServer.scala dcfca3f4156957b0006ba9bfdeca848c115825a8
core/src/main/scala/kafka/server/KafkaConfig.scala b0506d4881176cd95914847c4275e428a8d2ca10
core/src/main/scala/kafka/server/KafkaRequestHandler.scala 871212bf9110ca95ca7d2109be070f69d0f90247
Diff: https://reviews.apache.org/r/20131/diff/
Testing
-------
Local test with 9 new producer performance and one handler thread: in 3 minutes, the io threads idle time drops to 10%, but after that remains vibrating, the network threads keeps dropping but much slower, only to about 85% in 3 minutes.
Adding to 15 producers, the io thread starts to drop again, until close to 2% where it starts vibrating again, whereas the network threads drops to about 80%.
Thanks,
Guozhang Wang
Re: Review Request 20131: Fix KAFKA-1353: incorporate Jun's comment round two
Posted by Guozhang Wang <gu...@linkedin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20131/
-----------------------------------------------------------
(Updated April 8, 2014, 9:56 p.m.)
Review request for kafka.
Bugs: KAFKA-1353
https://issues.apache.org/jira/browse/KAFKA-1353
Repository: kafka
Description
-------
Remove unused imports
KAFKA-1353.v3
KAFKA-1353.v2
KAFKA-1353.v1
Diffs
-----
core/src/main/scala/kafka/network/RequestChannel.scala a6ec970d72fc8adc56bc6e200d1ff5bbf0d2b748
core/src/main/scala/kafka/network/SocketServer.scala dcfca3f4156957b0006ba9bfdeca848c115825a8
core/src/main/scala/kafka/server/KafkaRequestHandler.scala 871212bf9110ca95ca7d2109be070f69d0f90247
Diff: https://reviews.apache.org/r/20131/diff/
Testing (updated)
-------
Local test with 9 new producer performance and one handler thread: in 3 minutes, the io threads idle time drops to 10%, but after that remains vibrating, the network threads keeps dropping but much slower, only to about 85% in 3 minutes.
Adding to 15 producers, the io thread starts to drop again, until close to 2% where it starts vibrating again, whereas the network threads drops to about 80%.
Thanks,
Guozhang Wang