You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Stephane Maarek <st...@simplemachines.com.au> on 2018/09/02 18:25:06 UTC
KAFKA-1194
Hi,
I've seen Dong has done some work on
https://issues.apache.org/jira/browse/KAFKA-7278 which is said from the
comments that it could have possibly fixed
https://issues.apache.org/jira/browse/KAFKA-1194.
I tested and it is unfortunately not the case...
I have posted in KAFKA-1194 a way to reproduce the issue in a deterministic
way:
===========
C:\kafka_2.11-2.1.0-SNAPSHOT>bin\windows\kafka-server-start.bat
config\server.properties
C:\kafka_2.11-2.1.0-SNAPSHOT>kafka-topics.bat --zookeeper 127.0.0.1:2181
--topic second_topic --create --partitions 3 --replication-factor 1
C:\kafka_2.11-2.1.0-SNAPSHOT>kafka-console-producer.bat --broker-list
127.0.0.1:9092 --topic second_topic
>hello
>world
>hello
>Terminate batch job (Y/N)? Y
C:\kafka_2.11-2.1.0-SNAPSHOT>kafka-topics.bat --zookeeper 127.0.0.1:2181
--topic second_topic --delete
============
I usually wouldn't push for any issues for windows to be fixed, but this
one is triggered from basic CLI usage and triggers a full broker failure
that can't be recovered.
It actually completely destroys the faith of Kafka newcomers learning using
Windows (which are 50% of the users I have in my course).
I'm happy to help out in any way I can to test any patch.
Thanks
Stephane
Re: KAFKA-1194
Posted by Stephane Maarek <st...@simplemachines.com.au>.
Sorry for the double email. I've compiled the fixes made by the other PR
and it fixes the Windows problem: https://github.com/apache/kafka/pull/5603
It triggers a few errors in the unit tests, which is probably due to the
order changes of which files are deleted. Let's continue the discussion
there?
On 3 September 2018 at 08:24, Stephane Maarek <
stephane@simplemachines.com.au> wrote:
> Hi Ismael, thanks for having a look!
>
> https://github.com/apache/kafka/pull/4947/files does not fix it for me.
>
> But this one does: https://github.com/apache/kafka/pull/4431
> It produces the following log: https://pastebin.com/NHeAcc6v
> As you can see the folders do get renamed, it gives the user a few odd
> WARNs, but I checked and the directories are fully gone nonetheless.
>
> I think the ideas of the PRs go back all along to
> https://github.com/apache/kafka/pull/154/files or
> https://github.com/apache/kafka/pull/1757
>
> I couldn't find how to map both PRs to the new Kafka codebase,
> unfortunately.
>
> On 3 September 2018 at 03:26, Ismael Juma <is...@juma.me.uk> wrote:
>
>> Hi Stephane,
>>
>> Does https://github.com/apache/kafka/pull/4947/files fix it for you?
>>
>> Ismael
>>
>> On Sun, Sep 2, 2018 at 11:25 AM Stephane Maarek <
>> stephane@simplemachines.com.au> wrote:
>>
>> > Hi,
>> >
>> > I've seen Dong has done some work on
>> > https://issues.apache.org/jira/browse/KAFKA-7278 which is said from the
>> > comments that it could have possibly fixed
>> > https://issues.apache.org/jira/browse/KAFKA-1194.
>> >
>> > I tested and it is unfortunately not the case...
>> > I have posted in KAFKA-1194 a way to reproduce the issue in a
>> deterministic
>> > way:
>> > ===========
>> > C:\kafka_2.11-2.1.0-SNAPSHOT>bin\windows\kafka-server-start.bat
>> > config\server.properties
>> >
>> > C:\kafka_2.11-2.1.0-SNAPSHOT>kafka-topics.bat --zookeeper
>> 127.0.0.1:2181
>> > --topic second_topic --create --partitions 3 --replication-factor 1
>> >
>> > C:\kafka_2.11-2.1.0-SNAPSHOT>kafka-console-producer.bat --broker-list
>> > 127.0.0.1:9092 --topic second_topic
>> > >hello
>> > >world
>> > >hello
>> > >Terminate batch job (Y/N)? Y
>> >
>> > C:\kafka_2.11-2.1.0-SNAPSHOT>kafka-topics.bat --zookeeper
>> 127.0.0.1:2181
>> > --topic second_topic --delete
>> > ============
>> >
>> >
>> > I usually wouldn't push for any issues for windows to be fixed, but this
>> > one is triggered from basic CLI usage and triggers a full broker failure
>> > that can't be recovered.
>> > It actually completely destroys the faith of Kafka newcomers learning
>> using
>> > Windows (which are 50% of the users I have in my course).
>> >
>> > I'm happy to help out in any way I can to test any patch.
>> >
>> > Thanks
>> > Stephane
>> >
>>
>
>
Re: KAFKA-1194
Posted by Stephane Maarek <st...@simplemachines.com.au>.
Hi Ismael, thanks for having a look!
https://github.com/apache/kafka/pull/4947/files does not fix it for me.
But this one does: https://github.com/apache/kafka/pull/4431
It produces the following log: https://pastebin.com/NHeAcc6v
As you can see the folders do get renamed, it gives the user a few odd
WARNs, but I checked and the directories are fully gone nonetheless.
I think the ideas of the PRs go back all along to https://github.com/apache/
kafka/pull/154/files or https://github.com/apache/kafka/pull/1757
I couldn't find how to map both PRs to the new Kafka codebase,
unfortunately.
On 3 September 2018 at 03:26, Ismael Juma <is...@juma.me.uk> wrote:
> Hi Stephane,
>
> Does https://github.com/apache/kafka/pull/4947/files fix it for you?
>
> Ismael
>
> On Sun, Sep 2, 2018 at 11:25 AM Stephane Maarek <
> stephane@simplemachines.com.au> wrote:
>
> > Hi,
> >
> > I've seen Dong has done some work on
> > https://issues.apache.org/jira/browse/KAFKA-7278 which is said from the
> > comments that it could have possibly fixed
> > https://issues.apache.org/jira/browse/KAFKA-1194.
> >
> > I tested and it is unfortunately not the case...
> > I have posted in KAFKA-1194 a way to reproduce the issue in a
> deterministic
> > way:
> > ===========
> > C:\kafka_2.11-2.1.0-SNAPSHOT>bin\windows\kafka-server-start.bat
> > config\server.properties
> >
> > C:\kafka_2.11-2.1.0-SNAPSHOT>kafka-topics.bat --zookeeper 127.0.0.1:2181
> > --topic second_topic --create --partitions 3 --replication-factor 1
> >
> > C:\kafka_2.11-2.1.0-SNAPSHOT>kafka-console-producer.bat --broker-list
> > 127.0.0.1:9092 --topic second_topic
> > >hello
> > >world
> > >hello
> > >Terminate batch job (Y/N)? Y
> >
> > C:\kafka_2.11-2.1.0-SNAPSHOT>kafka-topics.bat --zookeeper 127.0.0.1:2181
> > --topic second_topic --delete
> > ============
> >
> >
> > I usually wouldn't push for any issues for windows to be fixed, but this
> > one is triggered from basic CLI usage and triggers a full broker failure
> > that can't be recovered.
> > It actually completely destroys the faith of Kafka newcomers learning
> using
> > Windows (which are 50% of the users I have in my course).
> >
> > I'm happy to help out in any way I can to test any patch.
> >
> > Thanks
> > Stephane
> >
>
Re: KAFKA-1194
Posted by Stephane Maarek <st...@simplemachines.com.au>.
Hi Manna
If you check the first email in this thread I was telling I already tested
the fix by Dong and that it unfortunately didn't work. I've been building
from trunk.
I've instead opened a PR based on various other people's fixes and it
passed Jenkins and also fixes the windows issues encountered in KAFKA-1194:
https://github.com/apache/kafka/pull/5603
Tested for topic deletion
Would love your pair of eyes on this too and testing if you can
Cheers
Stephane
On Mon., 3 Sep. 2018, 11:42 am M. Manna, <ma...@gmail.com> wrote:
> Hi Stephan,
>
> https://jira.apache.org/jira/browse/KAFKA-7278
>
> This might (according to Dong) be a better fix for the issue. Could you
> kindly apply to patch to your kafka (assuming that your version is
> reasonably close to the latest), and see if that helps?
>
> Regards,
>
> On Mon, 3 Sep 2018 at 02:27, Ismael Juma <is...@juma.me.uk> wrote:
>
> > Hi Stephane,
> >
> > Does https://github.com/apache/kafka/pull/4947/files fix it for you?
> >
> > Ismael
> >
> > On Sun, Sep 2, 2018 at 11:25 AM Stephane Maarek <
> > stephane@simplemachines.com.au> wrote:
> >
> > > Hi,
> > >
> > > I've seen Dong has done some work on
> > > https://issues.apache.org/jira/browse/KAFKA-7278 which is said from
> the
> > > comments that it could have possibly fixed
> > > https://issues.apache.org/jira/browse/KAFKA-1194.
> > >
> > > I tested and it is unfortunately not the case...
> > > I have posted in KAFKA-1194 a way to reproduce the issue in a
> > deterministic
> > > way:
> > > ===========
> > > C:\kafka_2.11-2.1.0-SNAPSHOT>bin\windows\kafka-server-start.bat
> > > config\server.properties
> > >
> > > C:\kafka_2.11-2.1.0-SNAPSHOT>kafka-topics.bat --zookeeper
> 127.0.0.1:2181
> > > --topic second_topic --create --partitions 3 --replication-factor 1
> > >
> > > C:\kafka_2.11-2.1.0-SNAPSHOT>kafka-console-producer.bat --broker-list
> > > 127.0.0.1:9092 --topic second_topic
> > > >hello
> > > >world
> > > >hello
> > > >Terminate batch job (Y/N)? Y
> > >
> > > C:\kafka_2.11-2.1.0-SNAPSHOT>kafka-topics.bat --zookeeper
> 127.0.0.1:2181
> > > --topic second_topic --delete
> > > ============
> > >
> > >
> > > I usually wouldn't push for any issues for windows to be fixed, but
> this
> > > one is triggered from basic CLI usage and triggers a full broker
> failure
> > > that can't be recovered.
> > > It actually completely destroys the faith of Kafka newcomers learning
> > using
> > > Windows (which are 50% of the users I have in my course).
> > >
> > > I'm happy to help out in any way I can to test any patch.
> > >
> > > Thanks
> > > Stephane
> > >
> >
>
Re: KAFKA-1194
Posted by Ismael Juma <is...@gmail.com>.
KAFKA-7278 doesn't fix the Windows issues.
Ismael
On Mon, 3 Sep 2018, 02:42 M. Manna, <ma...@gmail.com> wrote:
> Hi Stephan,
>
> https://jira.apache.org/jira/browse/KAFKA-7278
>
> This might (according to Dong) be a better fix for the issue. Could you
> kindly apply to patch to your kafka (assuming that your version is
> reasonably close to the latest), and see if that helps?
>
> Regards,
>
> On Mon, 3 Sep 2018 at 02:27, Ismael Juma <is...@juma.me.uk> wrote:
>
> > Hi Stephane,
> >
> > Does https://github.com/apache/kafka/pull/4947/files fix it for you?
> >
> > Ismael
> >
> > On Sun, Sep 2, 2018 at 11:25 AM Stephane Maarek <
> > stephane@simplemachines.com.au> wrote:
> >
> > > Hi,
> > >
> > > I've seen Dong has done some work on
> > > https://issues.apache.org/jira/browse/KAFKA-7278 which is said from
> the
> > > comments that it could have possibly fixed
> > > https://issues.apache.org/jira/browse/KAFKA-1194.
> > >
> > > I tested and it is unfortunately not the case...
> > > I have posted in KAFKA-1194 a way to reproduce the issue in a
> > deterministic
> > > way:
> > > ===========
> > > C:\kafka_2.11-2.1.0-SNAPSHOT>bin\windows\kafka-server-start.bat
> > > config\server.properties
> > >
> > > C:\kafka_2.11-2.1.0-SNAPSHOT>kafka-topics.bat --zookeeper
> 127.0.0.1:2181
> > > --topic second_topic --create --partitions 3 --replication-factor 1
> > >
> > > C:\kafka_2.11-2.1.0-SNAPSHOT>kafka-console-producer.bat --broker-list
> > > 127.0.0.1:9092 --topic second_topic
> > > >hello
> > > >world
> > > >hello
> > > >Terminate batch job (Y/N)? Y
> > >
> > > C:\kafka_2.11-2.1.0-SNAPSHOT>kafka-topics.bat --zookeeper
> 127.0.0.1:2181
> > > --topic second_topic --delete
> > > ============
> > >
> > >
> > > I usually wouldn't push for any issues for windows to be fixed, but
> this
> > > one is triggered from basic CLI usage and triggers a full broker
> failure
> > > that can't be recovered.
> > > It actually completely destroys the faith of Kafka newcomers learning
> > using
> > > Windows (which are 50% of the users I have in my course).
> > >
> > > I'm happy to help out in any way I can to test any patch.
> > >
> > > Thanks
> > > Stephane
> > >
> >
>
Re: KAFKA-1194
Posted by "M. Manna" <ma...@gmail.com>.
Hi Stephan,
https://jira.apache.org/jira/browse/KAFKA-7278
This might (according to Dong) be a better fix for the issue. Could you
kindly apply to patch to your kafka (assuming that your version is
reasonably close to the latest), and see if that helps?
Regards,
On Mon, 3 Sep 2018 at 02:27, Ismael Juma <is...@juma.me.uk> wrote:
> Hi Stephane,
>
> Does https://github.com/apache/kafka/pull/4947/files fix it for you?
>
> Ismael
>
> On Sun, Sep 2, 2018 at 11:25 AM Stephane Maarek <
> stephane@simplemachines.com.au> wrote:
>
> > Hi,
> >
> > I've seen Dong has done some work on
> > https://issues.apache.org/jira/browse/KAFKA-7278 which is said from the
> > comments that it could have possibly fixed
> > https://issues.apache.org/jira/browse/KAFKA-1194.
> >
> > I tested and it is unfortunately not the case...
> > I have posted in KAFKA-1194 a way to reproduce the issue in a
> deterministic
> > way:
> > ===========
> > C:\kafka_2.11-2.1.0-SNAPSHOT>bin\windows\kafka-server-start.bat
> > config\server.properties
> >
> > C:\kafka_2.11-2.1.0-SNAPSHOT>kafka-topics.bat --zookeeper 127.0.0.1:2181
> > --topic second_topic --create --partitions 3 --replication-factor 1
> >
> > C:\kafka_2.11-2.1.0-SNAPSHOT>kafka-console-producer.bat --broker-list
> > 127.0.0.1:9092 --topic second_topic
> > >hello
> > >world
> > >hello
> > >Terminate batch job (Y/N)? Y
> >
> > C:\kafka_2.11-2.1.0-SNAPSHOT>kafka-topics.bat --zookeeper 127.0.0.1:2181
> > --topic second_topic --delete
> > ============
> >
> >
> > I usually wouldn't push for any issues for windows to be fixed, but this
> > one is triggered from basic CLI usage and triggers a full broker failure
> > that can't be recovered.
> > It actually completely destroys the faith of Kafka newcomers learning
> using
> > Windows (which are 50% of the users I have in my course).
> >
> > I'm happy to help out in any way I can to test any patch.
> >
> > Thanks
> > Stephane
> >
>
Re: KAFKA-1194
Posted by Ismael Juma <is...@juma.me.uk>.
Hi Stephane,
Does https://github.com/apache/kafka/pull/4947/files fix it for you?
Ismael
On Sun, Sep 2, 2018 at 11:25 AM Stephane Maarek <
stephane@simplemachines.com.au> wrote:
> Hi,
>
> I've seen Dong has done some work on
> https://issues.apache.org/jira/browse/KAFKA-7278 which is said from the
> comments that it could have possibly fixed
> https://issues.apache.org/jira/browse/KAFKA-1194.
>
> I tested and it is unfortunately not the case...
> I have posted in KAFKA-1194 a way to reproduce the issue in a deterministic
> way:
> ===========
> C:\kafka_2.11-2.1.0-SNAPSHOT>bin\windows\kafka-server-start.bat
> config\server.properties
>
> C:\kafka_2.11-2.1.0-SNAPSHOT>kafka-topics.bat --zookeeper 127.0.0.1:2181
> --topic second_topic --create --partitions 3 --replication-factor 1
>
> C:\kafka_2.11-2.1.0-SNAPSHOT>kafka-console-producer.bat --broker-list
> 127.0.0.1:9092 --topic second_topic
> >hello
> >world
> >hello
> >Terminate batch job (Y/N)? Y
>
> C:\kafka_2.11-2.1.0-SNAPSHOT>kafka-topics.bat --zookeeper 127.0.0.1:2181
> --topic second_topic --delete
> ============
>
>
> I usually wouldn't push for any issues for windows to be fixed, but this
> one is triggered from basic CLI usage and triggers a full broker failure
> that can't be recovered.
> It actually completely destroys the faith of Kafka newcomers learning using
> Windows (which are 50% of the users I have in my course).
>
> I'm happy to help out in any way I can to test any patch.
>
> Thanks
> Stephane
>