You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Jun Rao <ju...@gmail.com> on 2014/03/21 02:01:02 UTC

Review Request 19510: Patch for KAFKA-1318

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19510/
-----------------------------------------------------------

Review request for kafka.


Bugs: KAFKA-1318
    https://issues.apache.org/jira/browse/KAFKA-1318


Repository: kafka


Description
-------

1. Changed the way to wait for producer to stop by checking the pid. 2. Added the log4j.properties for migration tool testsuite. 3. Passed in the right log4j property file for the broker. 4. Removed topic creating in migration tool test script since it's trying to run the 0.8 topic creation script on the 0.7 broker. Instead, increased retry.backoff.ms in migration producer to avoid data loss on auto created topic. 5. Improved README.


Diffs
-----

  system_test/README.txt 87937ecd7d637c62f2d401f54f1d669ac0d13f0a 
  system_test/migration_tool_testsuite/0.7/config/log4j.properties PRE-CREATION 
  system_test/migration_tool_testsuite/config/migration_producer.properties 17b5928a1213fc0cae7cd94cd214323ccca61d48 
  system_test/migration_tool_testsuite/migration_tool_test.py 2386a5884fcf86bac282ef28f07a3af740185f7f 
  system_test/utils/kafka_system_test_utils.py 35f2d1b0464edb738cd8e01f8654913776c49459 

Diff: https://reviews.apache.org/r/19510/diff/


Testing
-------


Thanks,

Jun Rao

Re: Review Request 19510: Patch for KAFKA-1318

Posted by Neha Narkhede <ne...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19510/#review38119
-----------------------------------------------------------

Ship it!


Ship It!

- Neha Narkhede


On March 21, 2014, 1:01 a.m., Jun Rao wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19510/
> -----------------------------------------------------------
> 
> (Updated March 21, 2014, 1:01 a.m.)
> 
> 
> Review request for kafka.
> 
> 
> Bugs: KAFKA-1318
>     https://issues.apache.org/jira/browse/KAFKA-1318
> 
> 
> Repository: kafka
> 
> 
> Description
> -------
> 
> 1. Changed the way to wait for producer to stop by checking the pid. 2. Added the log4j.properties for migration tool testsuite. 3. Passed in the right log4j property file for the broker. 4. Removed topic creating in migration tool test script since it's trying to run the 0.8 topic creation script on the 0.7 broker. Instead, increased retry.backoff.ms in migration producer to avoid data loss on auto created topic. 5. Improved README.
> 
> 
> Diffs
> -----
> 
>   system_test/README.txt 87937ecd7d637c62f2d401f54f1d669ac0d13f0a 
>   system_test/migration_tool_testsuite/0.7/config/log4j.properties PRE-CREATION 
>   system_test/migration_tool_testsuite/config/migration_producer.properties 17b5928a1213fc0cae7cd94cd214323ccca61d48 
>   system_test/migration_tool_testsuite/migration_tool_test.py 2386a5884fcf86bac282ef28f07a3af740185f7f 
>   system_test/utils/kafka_system_test_utils.py 35f2d1b0464edb738cd8e01f8654913776c49459 
> 
> Diff: https://reviews.apache.org/r/19510/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Jun Rao
> 
>

Re: Review Request 19510: Patch for KAFKA-1318-v2

Posted by Timothy Chen <tn...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19510/#review39070
-----------------------------------------------------------


Ah found it! Was searching for popen earlier.

- Timothy Chen


On March 31, 2014, 5:32 p.m., Jun Rao wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19510/
> -----------------------------------------------------------
> 
> (Updated March 31, 2014, 5:32 p.m.)
> 
> 
> Review request for kafka.
> 
> 
> Bugs: KAFKA-1318 and KAFKA-1318-v2
>     https://issues.apache.org/jira/browse/KAFKA-1318
>     https://issues.apache.org/jira/browse/KAFKA-1318-v2
> 
> 
> Repository: kafka
> 
> 
> Description
> -------
> 
> 1. Used Popen.communicate to wait for producer to stop. 2. Added the log4j.properties for migration tool testsuite. 3. Passed in the right log4j property file for the broker. 4. Removed topic creating in migration tool test script since it's trying to run the 0.8 topic creation script on the 0.7 broker. Instead, increased retry.backoff.ms in migration producer to avoid data loss on auto created topic. 5. Improved README.
> 
> 
> Diffs
> -----
> 
>   system_test/README.txt 87937ecd7d637c62f2d401f54f1d669ac0d13f0a 
>   system_test/migration_tool_testsuite/0.7/config/log4j.properties PRE-CREATION 
>   system_test/migration_tool_testsuite/config/migration_producer.properties 17b5928a1213fc0cae7cd94cd214323ccca61d48 
>   system_test/migration_tool_testsuite/migration_tool_test.py 2386a5884fcf86bac282ef28f07a3af740185f7f 
>   system_test/utils/kafka_system_test_utils.py 35f2d1b0464edb738cd8e01f8654913776c49459 
> 
> Diff: https://reviews.apache.org/r/19510/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Jun Rao
> 
>

Re: Review Request 19510: Patch for KAFKA-1318-v2

Posted by Jun Rao <ju...@gmail.com>.

> On March 31, 2014, 5:36 p.m., Timothy Chen wrote:
> > Not sure if I'm missing something here, but I don't see any Popen.communicate changes in this rb?

It is there. Search for subproc.communicate() in system_test/utils/kafka_system_test_utils.py.


- Jun


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19510/#review39068
-----------------------------------------------------------


On March 31, 2014, 5:32 p.m., Jun Rao wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19510/
> -----------------------------------------------------------
> 
> (Updated March 31, 2014, 5:32 p.m.)
> 
> 
> Review request for kafka.
> 
> 
> Bugs: KAFKA-1318 and KAFKA-1318-v2
>     https://issues.apache.org/jira/browse/KAFKA-1318
>     https://issues.apache.org/jira/browse/KAFKA-1318-v2
> 
> 
> Repository: kafka
> 
> 
> Description
> -------
> 
> 1. Used Popen.communicate to wait for producer to stop. 2. Added the log4j.properties for migration tool testsuite. 3. Passed in the right log4j property file for the broker. 4. Removed topic creating in migration tool test script since it's trying to run the 0.8 topic creation script on the 0.7 broker. Instead, increased retry.backoff.ms in migration producer to avoid data loss on auto created topic. 5. Improved README.
> 
> 
> Diffs
> -----
> 
>   system_test/README.txt 87937ecd7d637c62f2d401f54f1d669ac0d13f0a 
>   system_test/migration_tool_testsuite/0.7/config/log4j.properties PRE-CREATION 
>   system_test/migration_tool_testsuite/config/migration_producer.properties 17b5928a1213fc0cae7cd94cd214323ccca61d48 
>   system_test/migration_tool_testsuite/migration_tool_test.py 2386a5884fcf86bac282ef28f07a3af740185f7f 
>   system_test/utils/kafka_system_test_utils.py 35f2d1b0464edb738cd8e01f8654913776c49459 
> 
> Diff: https://reviews.apache.org/r/19510/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Jun Rao
> 
>

Re: Review Request 19510: Patch for KAFKA-1318-v2

Posted by Timothy Chen <tn...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19510/#review39068
-----------------------------------------------------------


Not sure if I'm missing something here, but I don't see any Popen.communicate changes in this rb?

- Timothy Chen


On March 31, 2014, 5:32 p.m., Jun Rao wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19510/
> -----------------------------------------------------------
> 
> (Updated March 31, 2014, 5:32 p.m.)
> 
> 
> Review request for kafka.
> 
> 
> Bugs: KAFKA-1318 and KAFKA-1318-v2
>     https://issues.apache.org/jira/browse/KAFKA-1318
>     https://issues.apache.org/jira/browse/KAFKA-1318-v2
> 
> 
> Repository: kafka
> 
> 
> Description
> -------
> 
> 1. Used Popen.communicate to wait for producer to stop. 2. Added the log4j.properties for migration tool testsuite. 3. Passed in the right log4j property file for the broker. 4. Removed topic creating in migration tool test script since it's trying to run the 0.8 topic creation script on the 0.7 broker. Instead, increased retry.backoff.ms in migration producer to avoid data loss on auto created topic. 5. Improved README.
> 
> 
> Diffs
> -----
> 
>   system_test/README.txt 87937ecd7d637c62f2d401f54f1d669ac0d13f0a 
>   system_test/migration_tool_testsuite/0.7/config/log4j.properties PRE-CREATION 
>   system_test/migration_tool_testsuite/config/migration_producer.properties 17b5928a1213fc0cae7cd94cd214323ccca61d48 
>   system_test/migration_tool_testsuite/migration_tool_test.py 2386a5884fcf86bac282ef28f07a3af740185f7f 
>   system_test/utils/kafka_system_test_utils.py 35f2d1b0464edb738cd8e01f8654913776c49459 
> 
> Diff: https://reviews.apache.org/r/19510/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Jun Rao
> 
>

Re: Review Request 19510: Patch for KAFKA-1318-v2

Posted by Guozhang Wang <gu...@linkedin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19510/#review39101
-----------------------------------------------------------

Ship it!


Ship It!

- Guozhang Wang


On March 31, 2014, 5:32 p.m., Jun Rao wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19510/
> -----------------------------------------------------------
> 
> (Updated March 31, 2014, 5:32 p.m.)
> 
> 
> Review request for kafka.
> 
> 
> Bugs: KAFKA-1318 and KAFKA-1318-v2
>     https://issues.apache.org/jira/browse/KAFKA-1318
>     https://issues.apache.org/jira/browse/KAFKA-1318-v2
> 
> 
> Repository: kafka
> 
> 
> Description
> -------
> 
> 1. Used Popen.communicate to wait for producer to stop. 2. Added the log4j.properties for migration tool testsuite. 3. Passed in the right log4j property file for the broker. 4. Removed topic creating in migration tool test script since it's trying to run the 0.8 topic creation script on the 0.7 broker. Instead, increased retry.backoff.ms in migration producer to avoid data loss on auto created topic. 5. Improved README.
> 
> 
> Diffs
> -----
> 
>   system_test/README.txt 87937ecd7d637c62f2d401f54f1d669ac0d13f0a 
>   system_test/migration_tool_testsuite/0.7/config/log4j.properties PRE-CREATION 
>   system_test/migration_tool_testsuite/config/migration_producer.properties 17b5928a1213fc0cae7cd94cd214323ccca61d48 
>   system_test/migration_tool_testsuite/migration_tool_test.py 2386a5884fcf86bac282ef28f07a3af740185f7f 
>   system_test/utils/kafka_system_test_utils.py 35f2d1b0464edb738cd8e01f8654913776c49459 
> 
> Diff: https://reviews.apache.org/r/19510/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Jun Rao
> 
>

Re: Review Request 19510: Patch for KAFKA-1318-v2

Posted by Neha Narkhede <ne...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19510/#review39171
-----------------------------------------------------------

Ship it!


Ship It!

- Neha Narkhede


On March 31, 2014, 5:32 p.m., Jun Rao wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19510/
> -----------------------------------------------------------
> 
> (Updated March 31, 2014, 5:32 p.m.)
> 
> 
> Review request for kafka.
> 
> 
> Bugs: KAFKA-1318 and KAFKA-1318-v2
>     https://issues.apache.org/jira/browse/KAFKA-1318
>     https://issues.apache.org/jira/browse/KAFKA-1318-v2
> 
> 
> Repository: kafka
> 
> 
> Description
> -------
> 
> 1. Used Popen.communicate to wait for producer to stop. 2. Added the log4j.properties for migration tool testsuite. 3. Passed in the right log4j property file for the broker. 4. Removed topic creating in migration tool test script since it's trying to run the 0.8 topic creation script on the 0.7 broker. Instead, increased retry.backoff.ms in migration producer to avoid data loss on auto created topic. 5. Improved README.
> 
> 
> Diffs
> -----
> 
>   system_test/README.txt 87937ecd7d637c62f2d401f54f1d669ac0d13f0a 
>   system_test/migration_tool_testsuite/0.7/config/log4j.properties PRE-CREATION 
>   system_test/migration_tool_testsuite/config/migration_producer.properties 17b5928a1213fc0cae7cd94cd214323ccca61d48 
>   system_test/migration_tool_testsuite/migration_tool_test.py 2386a5884fcf86bac282ef28f07a3af740185f7f 
>   system_test/utils/kafka_system_test_utils.py 35f2d1b0464edb738cd8e01f8654913776c49459 
> 
> Diff: https://reviews.apache.org/r/19510/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Jun Rao
> 
>

Re: Review Request 19510: Patch for KAFKA-1318-v2

Posted by Jun Rao <ju...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19510/
-----------------------------------------------------------

(Updated March 31, 2014, 5:32 p.m.)


Review request for kafka.


Summary (updated)
-----------------

Patch for KAFKA-1318-v2


Bugs: KAFKA-1318 and KAFKA-1318-v2
    https://issues.apache.org/jira/browse/KAFKA-1318
    https://issues.apache.org/jira/browse/KAFKA-1318-v2


Repository: kafka


Description (updated)
-------

1. Used Popen.communicate to wait for producer to stop. 2. Added the log4j.properties for migration tool testsuite. 3. Passed in the right log4j property file for the broker. 4. Removed topic creating in migration tool test script since it's trying to run the 0.8 topic creation script on the 0.7 broker. Instead, increased retry.backoff.ms in migration producer to avoid data loss on auto created topic. 5. Improved README.


Diffs (updated)
-----

  system_test/README.txt 87937ecd7d637c62f2d401f54f1d669ac0d13f0a 
  system_test/migration_tool_testsuite/0.7/config/log4j.properties PRE-CREATION 
  system_test/migration_tool_testsuite/config/migration_producer.properties 17b5928a1213fc0cae7cd94cd214323ccca61d48 
  system_test/migration_tool_testsuite/migration_tool_test.py 2386a5884fcf86bac282ef28f07a3af740185f7f 
  system_test/utils/kafka_system_test_utils.py 35f2d1b0464edb738cd8e01f8654913776c49459 

Diff: https://reviews.apache.org/r/19510/diff/


Testing
-------


Thanks,

Jun Rao

Re: Review Request 19510: Patch for KAFKA-1318

Posted by Guozhang Wang <gu...@linkedin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19510/#review38028
-----------------------------------------------------------

Ship it!



system_test/utils/kafka_system_test_utils.py
<https://reviews.apache.org/r/19510/#comment69963>

    Do we need to specify "comm=" here?


- Guozhang Wang


On March 21, 2014, 1:01 a.m., Jun Rao wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19510/
> -----------------------------------------------------------
> 
> (Updated March 21, 2014, 1:01 a.m.)
> 
> 
> Review request for kafka.
> 
> 
> Bugs: KAFKA-1318
>     https://issues.apache.org/jira/browse/KAFKA-1318
> 
> 
> Repository: kafka
> 
> 
> Description
> -------
> 
> 1. Changed the way to wait for producer to stop by checking the pid. 2. Added the log4j.properties for migration tool testsuite. 3. Passed in the right log4j property file for the broker. 4. Removed topic creating in migration tool test script since it's trying to run the 0.8 topic creation script on the 0.7 broker. Instead, increased retry.backoff.ms in migration producer to avoid data loss on auto created topic. 5. Improved README.
> 
> 
> Diffs
> -----
> 
>   system_test/README.txt 87937ecd7d637c62f2d401f54f1d669ac0d13f0a 
>   system_test/migration_tool_testsuite/0.7/config/log4j.properties PRE-CREATION 
>   system_test/migration_tool_testsuite/config/migration_producer.properties 17b5928a1213fc0cae7cd94cd214323ccca61d48 
>   system_test/migration_tool_testsuite/migration_tool_test.py 2386a5884fcf86bac282ef28f07a3af740185f7f 
>   system_test/utils/kafka_system_test_utils.py 35f2d1b0464edb738cd8e01f8654913776c49459 
> 
> Diff: https://reviews.apache.org/r/19510/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Jun Rao
> 
>