You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Geoffrey Anderson <ge...@confluent.io> on 2015/06/11 03:10:29 UTC
[VOTE] KIP-25 System test improvements
Hi Kafka,
After a few rounds of discussion on KIP-25, there doesn't seem to be
opposition, so I'd like to propose a vote.
Thanks,
Geoff
On Mon, Jun 8, 2015 at 10:56 PM, Geoffrey Anderson <ge...@confluent.io>
wrote:
> Hi KIP-25 thread,
>
> I consolidated some of the questions from this thread and elsewhere.
>
> Q: Can we see a map of what system-test currently tests, which ones we
> want to replace and JIRAs for replacing?
> A: Initial draft here:
> https://cwiki.apache.org/confluence/display/KAFKA/Roadmap+-+port+existing+system+tests
>
> Q: Will ducktape be maintained separately as a github repo?
> A: Yes https://github.com/confluentinc/ducktape
>
> Q: How easy is viewing the test results and logs, how will test output be
> structured?
> A: Hierarchical structure as outlined here:
> https://github.com/confluentinc/ducktape/wiki/Design-overview#output
>
> Q: Does it support code coverage? If not, how easy/ difficult would it be
> to support?
> A: It does not, and we have no immediate plans to support this. Difficulty
> unclear.
>
> Q: It would be nice if each Kafka version that we release will also
> have a separate "tests" artifact that users can download, untar and easily
> run against a Kafka cluster of the same version.
> A: This seems reasonable and not too much extra work. Definitely open to
> discussion on this.
>
> Q: Why not share running services across multiple tests?
> A: Prefer to optimize for simplicity and correctness over what might be a
> questionable improvement in run-time.
>
> Q: Are regressions - in the road map?
> A: yes
>
> Q: Are Jepsen style tests involving network failures in the road map?
> A: yes
>
> Thanks much,
> Geoff
>
>
>
Re: [VOTE] KIP-25 System test improvements
Posted by Geoffrey Anderson <ge...@confluent.io>.
Hi Gwen,
Yes, it looks like it passed with a lazy majority :)
I should be submitting a patch with a few initial tests today or tomorrow.
Cheers,
Geoff
On Mon, Jun 15, 2015 at 2:10 PM, Gwen Shapira <gs...@cloudera.com> wrote:
> I counted 3 binding votes (Jun, Neha, Joe) and two non-binding (Ewen and
> I).
> So I assume the KIP passed :)
>
> What's the next step? Are we expecting a patch with few initial tests soon?
>
> On Thu, Jun 11, 2015 at 5:41 PM, Jun Rao <ju...@confluent.io> wrote:
> > +1
> >
> > Thanks,
> >
> > Jun
> >
> > On Wed, Jun 10, 2015 at 6:10 PM, Geoffrey Anderson <ge...@confluent.io>
> > wrote:
> >
> >> Hi Kafka,
> >>
> >> After a few rounds of discussion on KIP-25, there doesn't seem to be
> >> opposition, so I'd like to propose a vote.
> >>
> >> Thanks,
> >> Geoff
> >>
> >> On Mon, Jun 8, 2015 at 10:56 PM, Geoffrey Anderson <ge...@confluent.io>
> >> wrote:
> >>
> >> > Hi KIP-25 thread,
> >> >
> >> > I consolidated some of the questions from this thread and elsewhere.
> >> >
> >> > Q: Can we see a map of what system-test currently tests, which ones we
> >> > want to replace and JIRAs for replacing?
> >> > A: Initial draft here:
> >> >
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/Roadmap+-+port+existing+system+tests
> >> >
> >> > Q: Will ducktape be maintained separately as a github repo?
> >> > A: Yes https://github.com/confluentinc/ducktape
> >> >
> >> > Q: How easy is viewing the test results and logs, how will test
> output be
> >> > structured?
> >> > A: Hierarchical structure as outlined here:
> >> > https://github.com/confluentinc/ducktape/wiki/Design-overview#output
> >> >
> >> > Q: Does it support code coverage? If not, how easy/ difficult would
> it be
> >> > to support?
> >> > A: It does not, and we have no immediate plans to support this.
> >> Difficulty
> >> > unclear.
> >> >
> >> > Q: It would be nice if each Kafka version that we release will also
> >> > have a separate "tests" artifact that users can download, untar and
> >> easily
> >> > run against a Kafka cluster of the same version.
> >> > A: This seems reasonable and not too much extra work. Definitely open
> to
> >> > discussion on this.
> >> >
> >> > Q: Why not share running services across multiple tests?
> >> > A: Prefer to optimize for simplicity and correctness over what might
> be a
> >> > questionable improvement in run-time.
> >> >
> >> > Q: Are regressions - in the road map?
> >> > A: yes
> >> >
> >> > Q: Are Jepsen style tests involving network failures in the road map?
> >> > A: yes
> >> >
> >> > Thanks much,
> >> > Geoff
> >> >
> >> >
> >> >
> >>
>
Re: [VOTE] KIP-25 System test improvements
Posted by Gwen Shapira <gs...@cloudera.com>.
I counted 3 binding votes (Jun, Neha, Joe) and two non-binding (Ewen and I).
So I assume the KIP passed :)
What's the next step? Are we expecting a patch with few initial tests soon?
On Thu, Jun 11, 2015 at 5:41 PM, Jun Rao <ju...@confluent.io> wrote:
> +1
>
> Thanks,
>
> Jun
>
> On Wed, Jun 10, 2015 at 6:10 PM, Geoffrey Anderson <ge...@confluent.io>
> wrote:
>
>> Hi Kafka,
>>
>> After a few rounds of discussion on KIP-25, there doesn't seem to be
>> opposition, so I'd like to propose a vote.
>>
>> Thanks,
>> Geoff
>>
>> On Mon, Jun 8, 2015 at 10:56 PM, Geoffrey Anderson <ge...@confluent.io>
>> wrote:
>>
>> > Hi KIP-25 thread,
>> >
>> > I consolidated some of the questions from this thread and elsewhere.
>> >
>> > Q: Can we see a map of what system-test currently tests, which ones we
>> > want to replace and JIRAs for replacing?
>> > A: Initial draft here:
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/Roadmap+-+port+existing+system+tests
>> >
>> > Q: Will ducktape be maintained separately as a github repo?
>> > A: Yes https://github.com/confluentinc/ducktape
>> >
>> > Q: How easy is viewing the test results and logs, how will test output be
>> > structured?
>> > A: Hierarchical structure as outlined here:
>> > https://github.com/confluentinc/ducktape/wiki/Design-overview#output
>> >
>> > Q: Does it support code coverage? If not, how easy/ difficult would it be
>> > to support?
>> > A: It does not, and we have no immediate plans to support this.
>> Difficulty
>> > unclear.
>> >
>> > Q: It would be nice if each Kafka version that we release will also
>> > have a separate "tests" artifact that users can download, untar and
>> easily
>> > run against a Kafka cluster of the same version.
>> > A: This seems reasonable and not too much extra work. Definitely open to
>> > discussion on this.
>> >
>> > Q: Why not share running services across multiple tests?
>> > A: Prefer to optimize for simplicity and correctness over what might be a
>> > questionable improvement in run-time.
>> >
>> > Q: Are regressions - in the road map?
>> > A: yes
>> >
>> > Q: Are Jepsen style tests involving network failures in the road map?
>> > A: yes
>> >
>> > Thanks much,
>> > Geoff
>> >
>> >
>> >
>>
Re: [VOTE] KIP-25 System test improvements
Posted by Jun Rao <ju...@confluent.io>.
+1
Thanks,
Jun
On Wed, Jun 10, 2015 at 6:10 PM, Geoffrey Anderson <ge...@confluent.io>
wrote:
> Hi Kafka,
>
> After a few rounds of discussion on KIP-25, there doesn't seem to be
> opposition, so I'd like to propose a vote.
>
> Thanks,
> Geoff
>
> On Mon, Jun 8, 2015 at 10:56 PM, Geoffrey Anderson <ge...@confluent.io>
> wrote:
>
> > Hi KIP-25 thread,
> >
> > I consolidated some of the questions from this thread and elsewhere.
> >
> > Q: Can we see a map of what system-test currently tests, which ones we
> > want to replace and JIRAs for replacing?
> > A: Initial draft here:
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Roadmap+-+port+existing+system+tests
> >
> > Q: Will ducktape be maintained separately as a github repo?
> > A: Yes https://github.com/confluentinc/ducktape
> >
> > Q: How easy is viewing the test results and logs, how will test output be
> > structured?
> > A: Hierarchical structure as outlined here:
> > https://github.com/confluentinc/ducktape/wiki/Design-overview#output
> >
> > Q: Does it support code coverage? If not, how easy/ difficult would it be
> > to support?
> > A: It does not, and we have no immediate plans to support this.
> Difficulty
> > unclear.
> >
> > Q: It would be nice if each Kafka version that we release will also
> > have a separate "tests" artifact that users can download, untar and
> easily
> > run against a Kafka cluster of the same version.
> > A: This seems reasonable and not too much extra work. Definitely open to
> > discussion on this.
> >
> > Q: Why not share running services across multiple tests?
> > A: Prefer to optimize for simplicity and correctness over what might be a
> > questionable improvement in run-time.
> >
> > Q: Are regressions - in the road map?
> > A: yes
> >
> > Q: Are Jepsen style tests involving network failures in the road map?
> > A: yes
> >
> > Thanks much,
> > Geoff
> >
> >
> >
>
Re: [VOTE] KIP-25 System test improvements
Posted by Jun Rao <ju...@confluent.io>.
+1
Thanks,
Jun
On Wed, Jun 10, 2015 at 6:10 PM, Geoffrey Anderson <ge...@confluent.io>
wrote:
> Hi Kafka,
>
> After a few rounds of discussion on KIP-25, there doesn't seem to be
> opposition, so I'd like to propose a vote.
>
> Thanks,
> Geoff
>
> On Mon, Jun 8, 2015 at 10:56 PM, Geoffrey Anderson <ge...@confluent.io>
> wrote:
>
> > Hi KIP-25 thread,
> >
> > I consolidated some of the questions from this thread and elsewhere.
> >
> > Q: Can we see a map of what system-test currently tests, which ones we
> > want to replace and JIRAs for replacing?
> > A: Initial draft here:
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Roadmap+-+port+existing+system+tests
> >
> > Q: Will ducktape be maintained separately as a github repo?
> > A: Yes https://github.com/confluentinc/ducktape
> >
> > Q: How easy is viewing the test results and logs, how will test output be
> > structured?
> > A: Hierarchical structure as outlined here:
> > https://github.com/confluentinc/ducktape/wiki/Design-overview#output
> >
> > Q: Does it support code coverage? If not, how easy/ difficult would it be
> > to support?
> > A: It does not, and we have no immediate plans to support this.
> Difficulty
> > unclear.
> >
> > Q: It would be nice if each Kafka version that we release will also
> > have a separate "tests" artifact that users can download, untar and
> easily
> > run against a Kafka cluster of the same version.
> > A: This seems reasonable and not too much extra work. Definitely open to
> > discussion on this.
> >
> > Q: Why not share running services across multiple tests?
> > A: Prefer to optimize for simplicity and correctness over what might be a
> > questionable improvement in run-time.
> >
> > Q: Are regressions - in the road map?
> > A: yes
> >
> > Q: Are Jepsen style tests involving network failures in the road map?
> > A: yes
> >
> > Thanks much,
> > Geoff
> >
> >
> >
>
Re: [VOTE] KIP-25 System test improvements
Posted by Ewen Cheslack-Postava <ew...@confluent.io>.
+1
-Ewen
On Thu, Jun 11, 2015 at 1:47 AM, Joe Stein <jo...@stealth.ly> wrote:
> +1
>
> ~ Joestein
> On Jun 10, 2015 10:21 PM, "Neha Narkhede" <ne...@confluent.io> wrote:
>
> > +1. Thanks Geoff!
> >
> >
> >
> >
> >
> > On Wed, Jun 10, 2015 at 6:20 PM -0700, "Gwen Shapira" <
> > gshapira@cloudera.com> wrote:
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > +1 (non-binding. Actually, since this is non-binding anyway, lets make
> > it +100. I'm so so excited about having a usable testing framework)
> >
> > On Wed, Jun 10, 2015 at 6:10 PM, Geoffrey Anderson wrote:
> > > Hi Kafka,
> > >
> > > After a few rounds of discussion on KIP-25, there doesn't seem to be
> > > opposition, so I'd like to propose a vote.
> > >
> > > Thanks,
> > > Geoff
> > >
> > > On Mon, Jun 8, 2015 at 10:56 PM, Geoffrey Anderson
> > > wrote:
> > >
> > >> Hi KIP-25 thread,
> > >>
> > >> I consolidated some of the questions from this thread and elsewhere.
> > >>
> > >> Q: Can we see a map of what system-test currently tests, which ones we
> > >> want to replace and JIRAs for replacing?
> > >> A: Initial draft here:
> > >>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Roadmap+-+port+existing+system+tests
> > >>
> > >> Q: Will ducktape be maintained separately as a github repo?
> > >> A: Yes https://github.com/confluentinc/ducktape
> > >>
> > >> Q: How easy is viewing the test results and logs, how will test output
> > be
> > >> structured?
> > >> A: Hierarchical structure as outlined here:
> > >> https://github.com/confluentinc/ducktape/wiki/Design-overview#output
> > >>
> > >> Q: Does it support code coverage? If not, how easy/ difficult would it
> > be
> > >> to support?
> > >> A: It does not, and we have no immediate plans to support this.
> > Difficulty
> > >> unclear.
> > >>
> > >> Q: It would be nice if each Kafka version that we release will also
> > >> have a separate "tests" artifact that users can download, untar and
> > easily
> > >> run against a Kafka cluster of the same version.
> > >> A: This seems reasonable and not too much extra work. Definitely open
> to
> > >> discussion on this.
> > >>
> > >> Q: Why not share running services across multiple tests?
> > >> A: Prefer to optimize for simplicity and correctness over what might
> be
> > a
> > >> questionable improvement in run-time.
> > >>
> > >> Q: Are regressions - in the road map?
> > >> A: yes
> > >>
> > >> Q: Are Jepsen style tests involving network failures in the road map?
> > >> A: yes
> > >>
> > >> Thanks much,
> > >> Geoff
> > >>
> > >>
> > >>
>
--
Thanks,
Ewen
Re: [VOTE] KIP-25 System test improvements
Posted by Joe Stein <jo...@stealth.ly>.
+1
~ Joestein
On Jun 10, 2015 10:21 PM, "Neha Narkhede" <ne...@confluent.io> wrote:
> +1. Thanks Geoff!
>
>
>
>
>
> On Wed, Jun 10, 2015 at 6:20 PM -0700, "Gwen Shapira" <
> gshapira@cloudera.com> wrote:
>
>
>
>
>
>
>
>
>
>
> +1 (non-binding. Actually, since this is non-binding anyway, lets make
> it +100. I'm so so excited about having a usable testing framework)
>
> On Wed, Jun 10, 2015 at 6:10 PM, Geoffrey Anderson wrote:
> > Hi Kafka,
> >
> > After a few rounds of discussion on KIP-25, there doesn't seem to be
> > opposition, so I'd like to propose a vote.
> >
> > Thanks,
> > Geoff
> >
> > On Mon, Jun 8, 2015 at 10:56 PM, Geoffrey Anderson
> > wrote:
> >
> >> Hi KIP-25 thread,
> >>
> >> I consolidated some of the questions from this thread and elsewhere.
> >>
> >> Q: Can we see a map of what system-test currently tests, which ones we
> >> want to replace and JIRAs for replacing?
> >> A: Initial draft here:
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/Roadmap+-+port+existing+system+tests
> >>
> >> Q: Will ducktape be maintained separately as a github repo?
> >> A: Yes https://github.com/confluentinc/ducktape
> >>
> >> Q: How easy is viewing the test results and logs, how will test output
> be
> >> structured?
> >> A: Hierarchical structure as outlined here:
> >> https://github.com/confluentinc/ducktape/wiki/Design-overview#output
> >>
> >> Q: Does it support code coverage? If not, how easy/ difficult would it
> be
> >> to support?
> >> A: It does not, and we have no immediate plans to support this.
> Difficulty
> >> unclear.
> >>
> >> Q: It would be nice if each Kafka version that we release will also
> >> have a separate "tests" artifact that users can download, untar and
> easily
> >> run against a Kafka cluster of the same version.
> >> A: This seems reasonable and not too much extra work. Definitely open to
> >> discussion on this.
> >>
> >> Q: Why not share running services across multiple tests?
> >> A: Prefer to optimize for simplicity and correctness over what might be
> a
> >> questionable improvement in run-time.
> >>
> >> Q: Are regressions - in the road map?
> >> A: yes
> >>
> >> Q: Are Jepsen style tests involving network failures in the road map?
> >> A: yes
> >>
> >> Thanks much,
> >> Geoff
> >>
> >>
> >>
Re: [VOTE] KIP-25 System test improvements
Posted by Neha Narkhede <ne...@confluent.io>.
+1. Thanks Geoff!
On Wed, Jun 10, 2015 at 6:20 PM -0700, "Gwen Shapira" <gs...@cloudera.com> wrote:
+1 (non-binding. Actually, since this is non-binding anyway, lets make
it +100. I'm so so excited about having a usable testing framework)
On Wed, Jun 10, 2015 at 6:10 PM, Geoffrey Anderson wrote:
> Hi Kafka,
>
> After a few rounds of discussion on KIP-25, there doesn't seem to be
> opposition, so I'd like to propose a vote.
>
> Thanks,
> Geoff
>
> On Mon, Jun 8, 2015 at 10:56 PM, Geoffrey Anderson
> wrote:
>
>> Hi KIP-25 thread,
>>
>> I consolidated some of the questions from this thread and elsewhere.
>>
>> Q: Can we see a map of what system-test currently tests, which ones we
>> want to replace and JIRAs for replacing?
>> A: Initial draft here:
>> https://cwiki.apache.org/confluence/display/KAFKA/Roadmap+-+port+existing+system+tests
>>
>> Q: Will ducktape be maintained separately as a github repo?
>> A: Yes https://github.com/confluentinc/ducktape
>>
>> Q: How easy is viewing the test results and logs, how will test output be
>> structured?
>> A: Hierarchical structure as outlined here:
>> https://github.com/confluentinc/ducktape/wiki/Design-overview#output
>>
>> Q: Does it support code coverage? If not, how easy/ difficult would it be
>> to support?
>> A: It does not, and we have no immediate plans to support this. Difficulty
>> unclear.
>>
>> Q: It would be nice if each Kafka version that we release will also
>> have a separate "tests" artifact that users can download, untar and easily
>> run against a Kafka cluster of the same version.
>> A: This seems reasonable and not too much extra work. Definitely open to
>> discussion on this.
>>
>> Q: Why not share running services across multiple tests?
>> A: Prefer to optimize for simplicity and correctness over what might be a
>> questionable improvement in run-time.
>>
>> Q: Are regressions - in the road map?
>> A: yes
>>
>> Q: Are Jepsen style tests involving network failures in the road map?
>> A: yes
>>
>> Thanks much,
>> Geoff
>>
>>
>>
Re: [VOTE] KIP-25 System test improvements
Posted by Gwen Shapira <gs...@cloudera.com>.
+1 (non-binding. Actually, since this is non-binding anyway, lets make
it +100. I'm so so excited about having a usable testing framework)
On Wed, Jun 10, 2015 at 6:10 PM, Geoffrey Anderson <ge...@confluent.io> wrote:
> Hi Kafka,
>
> After a few rounds of discussion on KIP-25, there doesn't seem to be
> opposition, so I'd like to propose a vote.
>
> Thanks,
> Geoff
>
> On Mon, Jun 8, 2015 at 10:56 PM, Geoffrey Anderson <ge...@confluent.io>
> wrote:
>
>> Hi KIP-25 thread,
>>
>> I consolidated some of the questions from this thread and elsewhere.
>>
>> Q: Can we see a map of what system-test currently tests, which ones we
>> want to replace and JIRAs for replacing?
>> A: Initial draft here:
>> https://cwiki.apache.org/confluence/display/KAFKA/Roadmap+-+port+existing+system+tests
>>
>> Q: Will ducktape be maintained separately as a github repo?
>> A: Yes https://github.com/confluentinc/ducktape
>>
>> Q: How easy is viewing the test results and logs, how will test output be
>> structured?
>> A: Hierarchical structure as outlined here:
>> https://github.com/confluentinc/ducktape/wiki/Design-overview#output
>>
>> Q: Does it support code coverage? If not, how easy/ difficult would it be
>> to support?
>> A: It does not, and we have no immediate plans to support this. Difficulty
>> unclear.
>>
>> Q: It would be nice if each Kafka version that we release will also
>> have a separate "tests" artifact that users can download, untar and easily
>> run against a Kafka cluster of the same version.
>> A: This seems reasonable and not too much extra work. Definitely open to
>> discussion on this.
>>
>> Q: Why not share running services across multiple tests?
>> A: Prefer to optimize for simplicity and correctness over what might be a
>> questionable improvement in run-time.
>>
>> Q: Are regressions - in the road map?
>> A: yes
>>
>> Q: Are Jepsen style tests involving network failures in the road map?
>> A: yes
>>
>> Thanks much,
>> Geoff
>>
>>
>>