MirrorMaker 2.0 Documentation

[Gwen Shapira <gw...@confluent.io>]

Hey,

I was surprised to discover that MirrorMaker 2.0 (part of 2.4
release), is not mentioned at all in our documentation
(https://kafka.apache.org/documentation/).

Ryanne Dolan was kind enough to point me to the documentation for this
feature (https://github.com/apache/kafka/blob/trunk/connect/mirror/README.md)
and mentioned that the Operations docs will be updated when MM1 is
deprecated.

IMO, this is a disservice to our users. If we already know that we
plan to deprecate MirrorMaker and replace it, we need to let users
know that an alternative exists, that they should start planning a
transition and how to use the cool new feature. And we need to do it
in the place where our users normally go to learn how to use our
product.

As it stands now, the README doesn't even show up on Google search
results, so discovering how to use MirrorMaker 2.0 is nearly
impossible. How can we expect to deprecate 1.0 if users can't find out
about 2.0?

In the past, we've documented new features before existing features
were deprecated (--broker-list and --bootstrap were documented before
--zookeeper was removed). I suggest we'll stick to this precedent.

Gwen

Re: MirrorMaker 2.0 Documentation

[Gwen Shapira <gw...@confluent.io>]

OK, awesome :)
I just wanted to make sure there are no objections before opening the
JIRA, since discussions are easier on the mailing list.

On Mon, Dec 23, 2019 at 5:01 PM Ryanne Dolan <ry...@gmail.com> wrote:
>
> Gwen, no one is hiding anything. Please follow up your JIRAs with PRs and
> I'm sure we can get the site updated.
>
> Ryanne
>
> On Mon, Dec 23, 2019, 8:38 AM Gwen Shapira <gw...@confluent.io> wrote:
>
> > There's nothing preventing us from updating the site-docs now to make them
> > match the 2.4 release better (I just opened few JIRA for additional docs).
> >
> > Is there a reason why we want to hide MirrorMaker 2.0 until next release?
> > For example, is it unstable? Do you expect interfaces or tools to
> > significantly evolve?
> >
> > On Mon, Dec 23, 2019, 4:33 PM Ryanne Dolan <ry...@gmail.com> wrote:
> >
> > > Hey Gwen, per the KIP, the depreciation plan will take a few releases.
> > > Copied here:
> > >
> > > Phase 1 (targeting next Apache Kafka release): All MirrorMaker 2.0 Java
> > > code is added to ./connect/mirror/.
> > >
> > > Phase 2 (subsequent release): Legacy MirrorMaker Scala code is
> > deprecated,
> > > but kept in place. Sample MM2 scripts and configuration files are added
> > to
> > > ./bin/ and ./config/.
> > >
> > > Phase 3 (subsequent release): Legacy MirrorMaker Scala code is removed
> > from
> > > Apache Kafka. A new ./bin/kafka-mirror-maker.sh script is provided which
> > > replaces and emulates the legacy script.
> > >
> > > Accordingly, let's plan to expand the documentation in the next release,
> > > and then remove the legacy documentation in Phase 3. At present, the
> > > existing documentation is still accurate.
> > >
> > > Ryanne
> > >
> > > On Mon, Dec 23, 2019, 2:25 AM Gwen Shapira <gw...@confluent.io> wrote:
> > >
> > > > Hey,
> > > >
> > > > I was surprised to discover that MirrorMaker 2.0 (part of 2.4
> > > > release), is not mentioned at all in our documentation
> > > > (https://kafka.apache.org/documentation/).
> > > >
> > > > Ryanne Dolan was kind enough to point me to the documentation for this
> > > > feature (
> > > > https://github.com/apache/kafka/blob/trunk/connect/mirror/README.md)
> > > > and mentioned that the Operations docs will be updated when MM1 is
> > > > deprecated.
> > > >
> > > > IMO, this is a disservice to our users. If we already know that we
> > > > plan to deprecate MirrorMaker and replace it, we need to let users
> > > > know that an alternative exists, that they should start planning a
> > > > transition and how to use the cool new feature. And we need to do it
> > > > in the place where our users normally go to learn how to use our
> > > > product.
> > > >
> > > > As it stands now, the README doesn't even show up on Google search
> > > > results, so discovering how to use MirrorMaker 2.0 is nearly
> > > > impossible. How can we expect to deprecate 1.0 if users can't find out
> > > > about 2.0?
> > > >
> > > > In the past, we've documented new features before existing features
> > > > were deprecated (--broker-list and --bootstrap were documented before
> > > > --zookeeper was removed). I suggest we'll stick to this precedent.
> > > >
> > > > Gwen
> > > >
> > >
> >

Re: MirrorMaker 2.0 Documentation

[Ryanne Dolan <ry...@gmail.com>]

Gwen, no one is hiding anything. Please follow up your JIRAs with PRs and
I'm sure we can get the site updated.

Ryanne

On Mon, Dec 23, 2019, 8:38 AM Gwen Shapira <gw...@confluent.io> wrote:

> There's nothing preventing us from updating the site-docs now to make them
> match the 2.4 release better (I just opened few JIRA for additional docs).
>
> Is there a reason why we want to hide MirrorMaker 2.0 until next release?
> For example, is it unstable? Do you expect interfaces or tools to
> significantly evolve?
>
> On Mon, Dec 23, 2019, 4:33 PM Ryanne Dolan <ry...@gmail.com> wrote:
>
> > Hey Gwen, per the KIP, the depreciation plan will take a few releases.
> > Copied here:
> >
> > Phase 1 (targeting next Apache Kafka release): All MirrorMaker 2.0 Java
> > code is added to ./connect/mirror/.
> >
> > Phase 2 (subsequent release): Legacy MirrorMaker Scala code is
> deprecated,
> > but kept in place. Sample MM2 scripts and configuration files are added
> to
> > ./bin/ and ./config/.
> >
> > Phase 3 (subsequent release): Legacy MirrorMaker Scala code is removed
> from
> > Apache Kafka. A new ./bin/kafka-mirror-maker.sh script is provided which
> > replaces and emulates the legacy script.
> >
> > Accordingly, let's plan to expand the documentation in the next release,
> > and then remove the legacy documentation in Phase 3. At present, the
> > existing documentation is still accurate.
> >
> > Ryanne
> >
> > On Mon, Dec 23, 2019, 2:25 AM Gwen Shapira <gw...@confluent.io> wrote:
> >
> > > Hey,
> > >
> > > I was surprised to discover that MirrorMaker 2.0 (part of 2.4
> > > release), is not mentioned at all in our documentation
> > > (https://kafka.apache.org/documentation/).
> > >
> > > Ryanne Dolan was kind enough to point me to the documentation for this
> > > feature (
> > > https://github.com/apache/kafka/blob/trunk/connect/mirror/README.md)
> > > and mentioned that the Operations docs will be updated when MM1 is
> > > deprecated.
> > >
> > > IMO, this is a disservice to our users. If we already know that we
> > > plan to deprecate MirrorMaker and replace it, we need to let users
> > > know that an alternative exists, that they should start planning a
> > > transition and how to use the cool new feature. And we need to do it
> > > in the place where our users normally go to learn how to use our
> > > product.
> > >
> > > As it stands now, the README doesn't even show up on Google search
> > > results, so discovering how to use MirrorMaker 2.0 is nearly
> > > impossible. How can we expect to deprecate 1.0 if users can't find out
> > > about 2.0?
> > >
> > > In the past, we've documented new features before existing features
> > > were deprecated (--broker-list and --bootstrap were documented before
> > > --zookeeper was removed). I suggest we'll stick to this precedent.
> > >
> > > Gwen
> > >
> >
>

Re: MirrorMaker 2.0 Documentation

[Gwen Shapira <gw...@confluent.io>]

There's nothing preventing us from updating the site-docs now to make them
match the 2.4 release better (I just opened few JIRA for additional docs).

Is there a reason why we want to hide MirrorMaker 2.0 until next release?
For example, is it unstable? Do you expect interfaces or tools to
significantly evolve?

On Mon, Dec 23, 2019, 4:33 PM Ryanne Dolan <ry...@gmail.com> wrote:

> Hey Gwen, per the KIP, the depreciation plan will take a few releases.
> Copied here:
>
> Phase 1 (targeting next Apache Kafka release): All MirrorMaker 2.0 Java
> code is added to ./connect/mirror/.
>
> Phase 2 (subsequent release): Legacy MirrorMaker Scala code is deprecated,
> but kept in place. Sample MM2 scripts and configuration files are added to
> ./bin/ and ./config/.
>
> Phase 3 (subsequent release): Legacy MirrorMaker Scala code is removed from
> Apache Kafka. A new ./bin/kafka-mirror-maker.sh script is provided which
> replaces and emulates the legacy script.
>
> Accordingly, let's plan to expand the documentation in the next release,
> and then remove the legacy documentation in Phase 3. At present, the
> existing documentation is still accurate.
>
> Ryanne
>
> On Mon, Dec 23, 2019, 2:25 AM Gwen Shapira <gw...@confluent.io> wrote:
>
> > Hey,
> >
> > I was surprised to discover that MirrorMaker 2.0 (part of 2.4
> > release), is not mentioned at all in our documentation
> > (https://kafka.apache.org/documentation/).
> >
> > Ryanne Dolan was kind enough to point me to the documentation for this
> > feature (
> > https://github.com/apache/kafka/blob/trunk/connect/mirror/README.md)
> > and mentioned that the Operations docs will be updated when MM1 is
> > deprecated.
> >
> > IMO, this is a disservice to our users. If we already know that we
> > plan to deprecate MirrorMaker and replace it, we need to let users
> > know that an alternative exists, that they should start planning a
> > transition and how to use the cool new feature. And we need to do it
> > in the place where our users normally go to learn how to use our
> > product.
> >
> > As it stands now, the README doesn't even show up on Google search
> > results, so discovering how to use MirrorMaker 2.0 is nearly
> > impossible. How can we expect to deprecate 1.0 if users can't find out
> > about 2.0?
> >
> > In the past, we've documented new features before existing features
> > were deprecated (--broker-list and --bootstrap were documented before
> > --zookeeper was removed). I suggest we'll stick to this precedent.
> >
> > Gwen
> >
>

Re: MirrorMaker 2.0 Documentation

[Ryanne Dolan <ry...@gmail.com>]

Hey Gwen, per the KIP, the depreciation plan will take a few releases.
Copied here:

Phase 1 (targeting next Apache Kafka release): All MirrorMaker 2.0 Java
code is added to ./connect/mirror/.

Phase 2 (subsequent release): Legacy MirrorMaker Scala code is deprecated,
but kept in place. Sample MM2 scripts and configuration files are added to
./bin/ and ./config/.

Phase 3 (subsequent release): Legacy MirrorMaker Scala code is removed from
Apache Kafka. A new ./bin/kafka-mirror-maker.sh script is provided which
replaces and emulates the legacy script.

Accordingly, let's plan to expand the documentation in the next release,
and then remove the legacy documentation in Phase 3. At present, the
existing documentation is still accurate.

Ryanne

On Mon, Dec 23, 2019, 2:25 AM Gwen Shapira <gw...@confluent.io> wrote:

> Hey,
>
> I was surprised to discover that MirrorMaker 2.0 (part of 2.4
> release), is not mentioned at all in our documentation
> (https://kafka.apache.org/documentation/).
>
> Ryanne Dolan was kind enough to point me to the documentation for this
> feature (
> https://github.com/apache/kafka/blob/trunk/connect/mirror/README.md)
> and mentioned that the Operations docs will be updated when MM1 is
> deprecated.
>
> IMO, this is a disservice to our users. If we already know that we
> plan to deprecate MirrorMaker and replace it, we need to let users
> know that an alternative exists, that they should start planning a
> transition and how to use the cool new feature. And we need to do it
> in the place where our users normally go to learn how to use our
> product.
>
> As it stands now, the README doesn't even show up on Google search
> results, so discovering how to use MirrorMaker 2.0 is nearly
> impossible. How can we expect to deprecate 1.0 if users can't find out
> about 2.0?
>
> In the past, we've documented new features before existing features
> were deprecated (--broker-list and --bootstrap were documented before
> --zookeeper was removed). I suggest we'll stick to this precedent.
>
> Gwen
>