You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Ben Davison <be...@7digital.com> on 2016/01/05 17:32:24 UTC
Migrate a topic which has no brokers
Hi All,
On our dev kafka environment our brokers went down with a topic on it, can
we just reassign the partitions to another broker?
Kafka 0.9
Thanks
Ben
--
This email, including attachments, is private and confidential. If you have
received this email in error please notify the sender and delete it from
your system. Emails are not secure and may contain viruses. No liability
can be accepted for viruses that might be transferred by this email or any
attachment. Any unauthorised copying of this message or unauthorised
distribution and publication of the information contained herein are
prohibited.
7digital Limited. Registered office: 69 Wilson Street, London EC2A 2BB.
Registered in England and Wales. Registered No. 04843573.
Re: Migrate a topic which has no brokers
Posted by Gwen Shapira <gw...@confluent.io>.
Hey,
You can modify a broker ID by taking it down, changing the id configuration
in the properties file and starting it.
Note that the broker will lose its existing partitions and take the
partitions of the broker it is replacing (i.e. a broker can't have two
identities at the same time), so you can do this only on brokers that
started but don't have data yet.
Gwen
On Jan 6, 2016 8:25 AM, "Ben Davison" <be...@7digital.com> wrote:
> Hi Gwen,
>
> Thanks for the suggestion, we are using Kafka 0.9's "auto broker ID"
> feature, is there a way to force a running kafka instance to take on a new
> broker ID?
>
> Thanks,
>
> Ben
>
> On Tue, Jan 5, 2016 at 6:19 PM, Gwen Shapira <gw...@confluent.io> wrote:
>
> > Stevo pointed you at the correct document for moving topics around.
> > However, if you lost a broker, by far the easiest way to recover is to
> > start a new broker and give it the same ID as the one that went down.
> >
> >
> >
> > On Tue, Jan 5, 2016 at 8:49 AM, Stevo Slavić <ss...@gmail.com> wrote:
> >
> > > Hello Ben,
> > >
> > > Yes, you can use apply different replica assignment. See related docs:
> > >
> > >
> >
> http://kafka.apache.org/documentation.html#basic_ops_increase_replication_factor
> > >
> > > Kind regards,
> > > Stevo Slavic.
> > >
> > > On Tue, Jan 5, 2016 at 5:32 PM, Ben Davison <be...@7digital.com>
> > > wrote:
> > >
> > > > Hi All,
> > > >
> > > > On our dev kafka environment our brokers went down with a topic on
> it,
> > > can
> > > > we just reassign the partitions to another broker?
> > > >
> > > > Kafka 0.9
> > > >
> > > > Thanks
> > > >
> > > > Ben
> > > >
> > > > --
> > > >
> > > >
> > > > This email, including attachments, is private and confidential. If
> you
> > > have
> > > > received this email in error please notify the sender and delete it
> > from
> > > > your system. Emails are not secure and may contain viruses. No
> > liability
> > > > can be accepted for viruses that might be transferred by this email
> or
> > > any
> > > > attachment. Any unauthorised copying of this message or unauthorised
> > > > distribution and publication of the information contained herein are
> > > > prohibited.
> > > >
> > > > 7digital Limited. Registered office: 69 Wilson Street, London EC2A
> 2BB.
> > > > Registered in England and Wales. Registered No. 04843573.
> > > >
> > >
> >
>
> --
>
>
> This email, including attachments, is private and confidential. If you have
> received this email in error please notify the sender and delete it from
> your system. Emails are not secure and may contain viruses. No liability
> can be accepted for viruses that might be transferred by this email or any
> attachment. Any unauthorised copying of this message or unauthorised
> distribution and publication of the information contained herein are
> prohibited.
>
> 7digital Limited. Registered office: 69 Wilson Street, London EC2A 2BB.
> Registered in England and Wales. Registered No. 04843573.
>
Re: Migrate a topic which has no brokers
Posted by Ben Davison <be...@7digital.com>.
Hi Gwen,
Thanks for the suggestion, we are using Kafka 0.9's "auto broker ID"
feature, is there a way to force a running kafka instance to take on a new
broker ID?
Thanks,
Ben
On Tue, Jan 5, 2016 at 6:19 PM, Gwen Shapira <gw...@confluent.io> wrote:
> Stevo pointed you at the correct document for moving topics around.
> However, if you lost a broker, by far the easiest way to recover is to
> start a new broker and give it the same ID as the one that went down.
>
>
>
> On Tue, Jan 5, 2016 at 8:49 AM, Stevo Slavić <ss...@gmail.com> wrote:
>
> > Hello Ben,
> >
> > Yes, you can use apply different replica assignment. See related docs:
> >
> >
> http://kafka.apache.org/documentation.html#basic_ops_increase_replication_factor
> >
> > Kind regards,
> > Stevo Slavic.
> >
> > On Tue, Jan 5, 2016 at 5:32 PM, Ben Davison <be...@7digital.com>
> > wrote:
> >
> > > Hi All,
> > >
> > > On our dev kafka environment our brokers went down with a topic on it,
> > can
> > > we just reassign the partitions to another broker?
> > >
> > > Kafka 0.9
> > >
> > > Thanks
> > >
> > > Ben
> > >
> > > --
> > >
> > >
> > > This email, including attachments, is private and confidential. If you
> > have
> > > received this email in error please notify the sender and delete it
> from
> > > your system. Emails are not secure and may contain viruses. No
> liability
> > > can be accepted for viruses that might be transferred by this email or
> > any
> > > attachment. Any unauthorised copying of this message or unauthorised
> > > distribution and publication of the information contained herein are
> > > prohibited.
> > >
> > > 7digital Limited. Registered office: 69 Wilson Street, London EC2A 2BB.
> > > Registered in England and Wales. Registered No. 04843573.
> > >
> >
>
--
This email, including attachments, is private and confidential. If you have
received this email in error please notify the sender and delete it from
your system. Emails are not secure and may contain viruses. No liability
can be accepted for viruses that might be transferred by this email or any
attachment. Any unauthorised copying of this message or unauthorised
distribution and publication of the information contained herein are
prohibited.
7digital Limited. Registered office: 69 Wilson Street, London EC2A 2BB.
Registered in England and Wales. Registered No. 04843573.
Re: Migrate a topic which has no brokers
Posted by Gwen Shapira <gw...@confluent.io>.
Stevo pointed you at the correct document for moving topics around.
However, if you lost a broker, by far the easiest way to recover is to
start a new broker and give it the same ID as the one that went down.
On Tue, Jan 5, 2016 at 8:49 AM, Stevo Slavić <ss...@gmail.com> wrote:
> Hello Ben,
>
> Yes, you can use apply different replica assignment. See related docs:
>
> http://kafka.apache.org/documentation.html#basic_ops_increase_replication_factor
>
> Kind regards,
> Stevo Slavic.
>
> On Tue, Jan 5, 2016 at 5:32 PM, Ben Davison <be...@7digital.com>
> wrote:
>
> > Hi All,
> >
> > On our dev kafka environment our brokers went down with a topic on it,
> can
> > we just reassign the partitions to another broker?
> >
> > Kafka 0.9
> >
> > Thanks
> >
> > Ben
> >
> > --
> >
> >
> > This email, including attachments, is private and confidential. If you
> have
> > received this email in error please notify the sender and delete it from
> > your system. Emails are not secure and may contain viruses. No liability
> > can be accepted for viruses that might be transferred by this email or
> any
> > attachment. Any unauthorised copying of this message or unauthorised
> > distribution and publication of the information contained herein are
> > prohibited.
> >
> > 7digital Limited. Registered office: 69 Wilson Street, London EC2A 2BB.
> > Registered in England and Wales. Registered No. 04843573.
> >
>
Re: Migrate a topic which has no brokers
Posted by Stevo Slavić <ss...@gmail.com>.
Hello Ben,
Yes, you can use apply different replica assignment. See related docs:
http://kafka.apache.org/documentation.html#basic_ops_increase_replication_factor
Kind regards,
Stevo Slavic.
On Tue, Jan 5, 2016 at 5:32 PM, Ben Davison <be...@7digital.com>
wrote:
> Hi All,
>
> On our dev kafka environment our brokers went down with a topic on it, can
> we just reassign the partitions to another broker?
>
> Kafka 0.9
>
> Thanks
>
> Ben
>
> --
>
>
> This email, including attachments, is private and confidential. If you have
> received this email in error please notify the sender and delete it from
> your system. Emails are not secure and may contain viruses. No liability
> can be accepted for viruses that might be transferred by this email or any
> attachment. Any unauthorised copying of this message or unauthorised
> distribution and publication of the information contained herein are
> prohibited.
>
> 7digital Limited. Registered office: 69 Wilson Street, London EC2A 2BB.
> Registered in England and Wales. Registered No. 04843573.
>