You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Chris Neal <cw...@gmail.com> on 2015/05/14 17:24:25 UTC
Partitions reassignment failed due to Partition reassignment
currently in progress for Map()
Hi All,
I am running: kafka_2.10-0.8.1.1, and when I run the
reassign-partitions.sh script, I get this:
Partitions reassignment failed due to Partition reassignment currently in
progress for Map(). Aborting operation
kafka.common.AdminCommandFailedException: Partition reassignment currently
in progress for Map(). Aborting operation
at
kafka.admin.ReassignPartitionsCommand.reassignPartitions(ReassignPartitionsCommand.scala:204)
at
kafka.admin.ReassignPartitionsCommand$.executeAssignment(ReassignPartitionsCommand.scala:124)
at
kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:49)
at
kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
I'm running it as:
/opt/kafka/kafka/bin/kafka-reassign-partitions.sh --zookeeper $ZOOKEEPER
--reassignment-json-file ./$1.json --execute
and my json file looks like:
{"version":1,
"partitions":[
{"topic":"perf_mytopic_raw", "partition":0,"replicas":[0,1]},
{"topic":"perf_mytopic_raw","partition":1,"replicas":[0,1]}]}
The empty Map() looks suspicious, and I found another post from about a
year ago where it was thought that this might be a bug? I looked through
JIRA and didn't see anything that looked like this issue was created or
fixed, so I wanted to ask again about it.
Any ideas why this might be failing? There is definitely not another
reassignment in progress. :)
Thanks very much for your time.
Chris
Re: Partitions reassignment failed due to Partition reassignment
currently in progress for Map()
Posted by Chris Neal <cw...@gmail.com>.
Perfect. Thank you very much!
Chris
On Sun, May 17, 2015 at 10:41 PM, Clark Haskins <cl...@kafka.guru> wrote:
> Yeah you can start using the reassignment tool again.
>
> It's likely some issue with the controller that put you in this state.
> Without gathering the logs it will be hard to tell.
>
> -Clark
>
> Sent from my iPhone
>
> > On May 17, 2015, at 3:42 PM, Chris Neal <cw...@gmail.com> wrote:
> >
> > Done, and things look clean in Zookeeper now. I'm guessing that it is
> safe
> > for me to try to increase the replication count on this topic again? :)
> >
> > Any idea what might have happened to put me in this state?
> > I had successfully increased probably 10 or so topics, but then I started
> > getting this message and nothing worked after that.
> >
> > Again, thank you so much for your time and knowledge. Very much
> > appreciated.
> > Chris
> >
> >> On Sun, May 17, 2015 at 2:20 PM, Clark Haskins <cl...@kafka.guru>
> wrote:
> >>
> >> No problem.
> >>
> >> Delete the reassign_partitions znode. Then do the controller again.
> >>
> >> The controller znode gets instantly re-created but with a different
> broker
> >> as the controller.
> >>
> >> If you do a get /controller you can see the values change
> >>
> >> -Clark
> >>
> >> Sent from my iPhone
> >>
> >>> On May 17, 2015, at 11:48 AM, Chris Neal <cw...@gmail.com> wrote:
> >>>
> >>> Gah. Sorry for the spamming replies.
> >>>
> >>> The reassign_partitions znode is also still there, and still
> empty....For
> >>> what that's worth. :)
> >>> Back to thinking that deleting the reassign_partitions znode is the
> right
> >>> way to go.
> >>>
> >>> Chris
> >>>
> >>>> On Sun, May 17, 2015 at 1:46 PM, Chris Neal <cw...@gmail.com> wrote:
> >>>>
> >>>> Ok, I might have lied. Timezone got me. I do see this in the Kafka
> >> logs,
> >>>> so maybe it did work?
> >>>>
> >>>> [2015-05-17 14:36:57,924] INFO 0 successfully elected as leader
> >>>> (kafka.server.ZookeeperLeaderElector)
> >>>> [2015-05-17 14:36:57,924] INFO 0 successfully elected as leader
> >>>> (kafka.server.ZookeeperLeaderElector)
> >>>> [2015-05-17 14:36:58,975] INFO New leader is 0
> >>>> (kafka.server.ZookeeperLeaderElector$LeaderChangeListener)
> >>>> [2015-05-17 14:36:58,975] INFO New leader is 0
> >>>> (kafka.server.ZookeeperLeaderElector$LeaderChangeListener)
> >>>>
> >>>> Chris
> >>>>
> >>>>> On Sun, May 17, 2015 at 1:43 PM, Chris Neal <cw...@gmail.com>
> wrote:
> >>>>>
> >>>>> Thank you again. :)
> >>>>>
> >>>>> I ran the delete, but I'm not sure if it did anything:
> >>>>>
> >>>>> root>
> >>
> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
> >>>>> -server myhost.mydomain.com:2181 delete /controller
> >>>>> Connecting to myhost.mydomain.com:2181
> >>>>>
> >>>>> WATCHER::
> >>>>>
> >>>>> WatchedEvent state:SyncConnected type:None path:null
> >>>>> root>
> >>>>>
> >>>>> I followed that up immediately by an "ls /", and the controller znode
> >> was
> >>>>> there. Either still there, or had already been re-created. There
> was
> >>>>> nothing in the Kafka logs indicating that anything had happened
> >> either, so
> >>>>> I'm thinking maybe I need to try and delete the znode partition
> >> manually?
> >>>>>
> >>>>> Or, should I have seen the controller znode disappear for a time?
> >>>>>
> >>>>> Very much appreciate your help!
> >>>>> Chris
> >>>>>
> >>>>>> On Sun, May 17, 2015 at 1:33 PM, Clark Haskins <cl...@kafka.guru>
> >> wrote:
> >>>>>>
> >>>>>> Yeah it's empty. So I would try and force a new controller election
> to
> >>>>>> see if that cleans up the znode.
> >>>>>>
> >>>>>> Delete /controller
> >>>>>>
> >>>>>> If that doesn't work you will have to delete the reassign_partitions
> >>>>>> znode manually
> >>>>>>
> >>>>>> -Clark
> >>>>>>
> >>>>>> Sent from my iPhone
> >>>>>>
> >>>>>>> On May 17, 2015, at 11:16 AM, Chris Neal <cw...@gmail.com> wrote:
> >>>>>>>
> >>>>>>> Sorry :)
> >>>>>>> Here you go:
> >>>>>>>
> >>>>>>> root>
> >>
> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
> >>>>>>> -server myhost.mydomain.com <http://myhost.mydomain.com:2181/
> >:2181
> >>>>>> get
> >>>>>>> /admin/reassign_partitions
> >>>>>>> Connecting to myhost.mydomain.com <
> http://myhost.mydomain.com:2181/
> >>>>>>> :2181
> >>>>>>> <snip>
> >>>>>>> WATCHER::
> >>>>>>>
> >>>>>>> WatchedEvent state:SyncConnected type:None path:null
> >>>>>>> {"version":1,"partitions":[]}
> >>>>>>> cZxid = 0x6001afde9
> >>>>>>> ctime = Wed Jan 21 18:37:40 UTC 2015
> >>>>>>> mZxid = 0x6001afde9
> >>>>>>> mtime = Wed Jan 21 18:37:40 UTC 2015
> >>>>>>> pZxid = 0x6001afde9
> >>>>>>> cversion = 0
> >>>>>>> dataVersion = 0
> >>>>>>> aclVersion = 0
> >>>>>>> ephemeralOwner = 0x0
> >>>>>>> dataLength = 29
> >>>>>>> numChildren = 0
> >>>>>>>
> >>>>>>> Thank you!
> >>>>>>> Chris
> >>>>>>>
> >>>>>>>> On Sun, May 17, 2015 at 12:58 PM, Clark Haskins <clark@kafka.guru
> >
> >>>>>> wrote:
> >>>>>>>>
> >>>>>>>> Do a get /admin/reassign_partitions
> >>>>>>>>
> >>>>>>>> -Clark
> >>>>>>>>
> >>>>>>>> Sent from my iPhone
> >>>>>>>>
> >>>>>>>>> On May 17, 2015, at 10:20 AM, Chris Neal <cw...@gmail.com>
> wrote:
> >>>>>>>>>
> >>>>>>>>> Sure thing :)
> >>>>>>>>> Hopefully I did this right. Somewhat of a Zookeeper noob.
> >>>>>>>>>
> >>>>>>>>> root>
> >>
> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
> >>>>>>>>> -server myhost.mydomain.com <http://myhost.mydomain.com:2181/
> >>> :2181
> >>>>>> ls
> >>>>>>>>> /admin/reassign_partitions
> >>>>>>>>> Connecting to myhost.mydomain.com <
> >> http://myhost.mydomain.com:2181/
> >>>>>>>>> :2181
> >>>>>>>>> <snip>
> >>>>>>>>> WATCHER::
> >>>>>>>>>
> >>>>>>>>> WatchedEvent state:SyncConnected type:None path:null
> >>>>>>>>> []
> >>>>>>>>>
> >>>>>>>>> Hope that is helpful :)
> >>>>>>>>> If this is not what you were asking for, please just let me know.
> >>>>>>>>> Thank you!
> >>>>>>>>> Chris
> >>>>>>>>>
> >>>>>>>>>> On Sun, May 17, 2015 at 12:17 PM, Clark Haskins
> <clark@kafka.guru
> >>>
> >>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> The reassign_partitions znode is the important one. Please paste
> >> the
> >>>>>>>>>> contents of it. That node should only exist while there is a
> >>>>>>>> reassignment
> >>>>>>>>>> in progress.
> >>>>>>>>>>
> >>>>>>>>>> You can probably fix this up by forcing a new controller to come
> >>>>>> online
> >>>>>>>> by
> >>>>>>>>>> deleting /controller
> >>>>>>>>>>
> >>>>>>>>>> -Clark
> >>>>>>>>>>
> >>>>>>>>>> Sent from my iPhone
> >>>>>>>>>>
> >>>>>>>>>>> On May 17, 2015, at 10:14 AM, Chris Neal <cw...@gmail.com>
> >> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> Hi Clark,
> >>>>>>>>>>>
> >>>>>>>>>>> Thank you for your reply! I do see that znode under /admin:
> >>>>>>>>>>>
> >>>>>>>>>>> root>
> >>
> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
> >>>>>>>>>>> -server myhost.mydomain.com:2181 ls /admin
> >>>>>>>>>>> Connecting to myhost.mydomain.com:2181
> >>>>>>>>>>> <snip>
> >>>>>>>>>>> WATCHER::
> >>>>>>>>>>>
> >>>>>>>>>>> WatchedEvent state:SyncConnected type:None path:null
> >>>>>>>>>>> [reassign_partitions, delete_topics]
> >>>>>>>>>>>
> >>>>>>>>>>> I'm not sure what this tells me though :)
> >>>>>>>>>>> Again, thanks for your time.
> >>>>>>>>>>> Chris
> >>>>>>>>>>>
> >>>>>>>>>>>> On Sun, May 17, 2015 at 12:20 AM, Clark Haskins
> >> <clark@kafka.guru
> >>>>>>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> Does the partition reassignment znode exist under /admin in
> >>>>>> zookeeper?
> >>>>>>>>>>>>
> >>>>>>>>>>>> -Clark
> >>>>>>>>>>>>
> >>>>>>>>>>>> Sent from my iPhone
> >>>>>>>>>>>>
> >>>>>>>>>>>>> On May 16, 2015, at 7:16 PM, Chris Neal <cw...@gmail.com>
> >>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Sorry for bumping my own thread. :S Just wanted to get it in
> >>>>>> front
> >>>>>>>> of
> >>>>>>>>>>>> some
> >>>>>>>>>>>>> eyes again!
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks for your time and help.
> >>>>>>>>>>>>> Chris
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Thu, May 14, 2015 at 10:24 AM, Chris Neal <
> >> cwneal@gmail.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I am running: kafka_2.10-0.8.1.1, and when I run the
> >>>>>>>>>>>>>> reassign-partitions.sh script, I get this:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Partitions reassignment failed due to Partition reassignment
> >>>>>>>> currently
> >>>>>>>>>>>> in
> >>>>>>>>>>>>>> progress for Map(). Aborting operation
> >>>>>>>>>>>>>> kafka.common.AdminCommandFailedException: Partition
> >> reassignment
> >>>>>>>>>>>> currently
> >>>>>>>>>>>>>> in progress for Map(). Aborting operation
> >>>>>>>>>>>>>> at
> >>
> kafka.admin.ReassignPartitionsCommand.reassignPartitions(ReassignPartitionsCommand.scala:204)
> >>>>>>>>>>>>>> at
> >>
> kafka.admin.ReassignPartitionsCommand$.executeAssignment(ReassignPartitionsCommand.scala:124)
> >>>>>>>>>>>>>> at
> >>
> kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:49)
> >>>>>>>>>>>>>> at
> >>
> kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I'm running it as:
> >>>>>>>>>>>>>> /opt/kafka/kafka/bin/kafka-reassign-partitions.sh
> --zookeeper
> >>>>>>>>>> $ZOOKEEPER
> >>>>>>>>>>>>>> --reassignment-json-file ./$1.json --execute
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> and my json file looks like:
> >>>>>>>>>>>>>> {"version":1,
> >>>>>>>>>>>>>> "partitions":[
> >>>>>>>>>>>>>> {"topic":"perf_mytopic_raw",
> "partition":0,"replicas":[0,1]},
> >>>>>>>>>>>>>>
> {"topic":"perf_mytopic_raw","partition":1,"replicas":[0,1]}]}
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> The empty Map() looks suspicious, and I found another post
> >> from
> >>>>>>>> about
> >>>>>>>>>> a
> >>>>>>>>>>>>>> year ago where it was thought that this might be a bug? I
> >>>>>> looked
> >>>>>>>>>>>> through
> >>>>>>>>>>>>>> JIRA and didn't see anything that looked like this issue was
> >>>>>> created
> >>>>>>>>>> or
> >>>>>>>>>>>>>> fixed, so I wanted to ask again about it.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Any ideas why this might be failing? There is definitely
> not
> >>>>>>>> another
> >>>>>>>>>>>>>> reassignment in progress. :)
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks very much for your time.
> >>>>>>>>>>>>>> Chris
> >>
>
Re: Partitions reassignment failed due to Partition reassignment currently in progress for Map()
Posted by Clark Haskins <cl...@kafka.guru>.
Yeah you can start using the reassignment tool again.
It's likely some issue with the controller that put you in this state. Without gathering the logs it will be hard to tell.
-Clark
Sent from my iPhone
> On May 17, 2015, at 3:42 PM, Chris Neal <cw...@gmail.com> wrote:
>
> Done, and things look clean in Zookeeper now. I'm guessing that it is safe
> for me to try to increase the replication count on this topic again? :)
>
> Any idea what might have happened to put me in this state?
> I had successfully increased probably 10 or so topics, but then I started
> getting this message and nothing worked after that.
>
> Again, thank you so much for your time and knowledge. Very much
> appreciated.
> Chris
>
>> On Sun, May 17, 2015 at 2:20 PM, Clark Haskins <cl...@kafka.guru> wrote:
>>
>> No problem.
>>
>> Delete the reassign_partitions znode. Then do the controller again.
>>
>> The controller znode gets instantly re-created but with a different broker
>> as the controller.
>>
>> If you do a get /controller you can see the values change
>>
>> -Clark
>>
>> Sent from my iPhone
>>
>>> On May 17, 2015, at 11:48 AM, Chris Neal <cw...@gmail.com> wrote:
>>>
>>> Gah. Sorry for the spamming replies.
>>>
>>> The reassign_partitions znode is also still there, and still empty....For
>>> what that's worth. :)
>>> Back to thinking that deleting the reassign_partitions znode is the right
>>> way to go.
>>>
>>> Chris
>>>
>>>> On Sun, May 17, 2015 at 1:46 PM, Chris Neal <cw...@gmail.com> wrote:
>>>>
>>>> Ok, I might have lied. Timezone got me. I do see this in the Kafka
>> logs,
>>>> so maybe it did work?
>>>>
>>>> [2015-05-17 14:36:57,924] INFO 0 successfully elected as leader
>>>> (kafka.server.ZookeeperLeaderElector)
>>>> [2015-05-17 14:36:57,924] INFO 0 successfully elected as leader
>>>> (kafka.server.ZookeeperLeaderElector)
>>>> [2015-05-17 14:36:58,975] INFO New leader is 0
>>>> (kafka.server.ZookeeperLeaderElector$LeaderChangeListener)
>>>> [2015-05-17 14:36:58,975] INFO New leader is 0
>>>> (kafka.server.ZookeeperLeaderElector$LeaderChangeListener)
>>>>
>>>> Chris
>>>>
>>>>> On Sun, May 17, 2015 at 1:43 PM, Chris Neal <cw...@gmail.com> wrote:
>>>>>
>>>>> Thank you again. :)
>>>>>
>>>>> I ran the delete, but I'm not sure if it did anything:
>>>>>
>>>>> root>
>> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
>>>>> -server myhost.mydomain.com:2181 delete /controller
>>>>> Connecting to myhost.mydomain.com:2181
>>>>>
>>>>> WATCHER::
>>>>>
>>>>> WatchedEvent state:SyncConnected type:None path:null
>>>>> root>
>>>>>
>>>>> I followed that up immediately by an "ls /", and the controller znode
>> was
>>>>> there. Either still there, or had already been re-created. There was
>>>>> nothing in the Kafka logs indicating that anything had happened
>> either, so
>>>>> I'm thinking maybe I need to try and delete the znode partition
>> manually?
>>>>>
>>>>> Or, should I have seen the controller znode disappear for a time?
>>>>>
>>>>> Very much appreciate your help!
>>>>> Chris
>>>>>
>>>>>> On Sun, May 17, 2015 at 1:33 PM, Clark Haskins <cl...@kafka.guru>
>> wrote:
>>>>>>
>>>>>> Yeah it's empty. So I would try and force a new controller election to
>>>>>> see if that cleans up the znode.
>>>>>>
>>>>>> Delete /controller
>>>>>>
>>>>>> If that doesn't work you will have to delete the reassign_partitions
>>>>>> znode manually
>>>>>>
>>>>>> -Clark
>>>>>>
>>>>>> Sent from my iPhone
>>>>>>
>>>>>>> On May 17, 2015, at 11:16 AM, Chris Neal <cw...@gmail.com> wrote:
>>>>>>>
>>>>>>> Sorry :)
>>>>>>> Here you go:
>>>>>>>
>>>>>>> root>
>> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
>>>>>>> -server myhost.mydomain.com <http://myhost.mydomain.com:2181/>:2181
>>>>>> get
>>>>>>> /admin/reassign_partitions
>>>>>>> Connecting to myhost.mydomain.com <http://myhost.mydomain.com:2181/
>>>>>>> :2181
>>>>>>> <snip>
>>>>>>> WATCHER::
>>>>>>>
>>>>>>> WatchedEvent state:SyncConnected type:None path:null
>>>>>>> {"version":1,"partitions":[]}
>>>>>>> cZxid = 0x6001afde9
>>>>>>> ctime = Wed Jan 21 18:37:40 UTC 2015
>>>>>>> mZxid = 0x6001afde9
>>>>>>> mtime = Wed Jan 21 18:37:40 UTC 2015
>>>>>>> pZxid = 0x6001afde9
>>>>>>> cversion = 0
>>>>>>> dataVersion = 0
>>>>>>> aclVersion = 0
>>>>>>> ephemeralOwner = 0x0
>>>>>>> dataLength = 29
>>>>>>> numChildren = 0
>>>>>>>
>>>>>>> Thank you!
>>>>>>> Chris
>>>>>>>
>>>>>>>> On Sun, May 17, 2015 at 12:58 PM, Clark Haskins <cl...@kafka.guru>
>>>>>> wrote:
>>>>>>>>
>>>>>>>> Do a get /admin/reassign_partitions
>>>>>>>>
>>>>>>>> -Clark
>>>>>>>>
>>>>>>>> Sent from my iPhone
>>>>>>>>
>>>>>>>>> On May 17, 2015, at 10:20 AM, Chris Neal <cw...@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>> Sure thing :)
>>>>>>>>> Hopefully I did this right. Somewhat of a Zookeeper noob.
>>>>>>>>>
>>>>>>>>> root>
>> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
>>>>>>>>> -server myhost.mydomain.com <http://myhost.mydomain.com:2181/
>>> :2181
>>>>>> ls
>>>>>>>>> /admin/reassign_partitions
>>>>>>>>> Connecting to myhost.mydomain.com <
>> http://myhost.mydomain.com:2181/
>>>>>>>>> :2181
>>>>>>>>> <snip>
>>>>>>>>> WATCHER::
>>>>>>>>>
>>>>>>>>> WatchedEvent state:SyncConnected type:None path:null
>>>>>>>>> []
>>>>>>>>>
>>>>>>>>> Hope that is helpful :)
>>>>>>>>> If this is not what you were asking for, please just let me know.
>>>>>>>>> Thank you!
>>>>>>>>> Chris
>>>>>>>>>
>>>>>>>>>> On Sun, May 17, 2015 at 12:17 PM, Clark Haskins <clark@kafka.guru
>>>
>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> The reassign_partitions znode is the important one. Please paste
>> the
>>>>>>>>>> contents of it. That node should only exist while there is a
>>>>>>>> reassignment
>>>>>>>>>> in progress.
>>>>>>>>>>
>>>>>>>>>> You can probably fix this up by forcing a new controller to come
>>>>>> online
>>>>>>>> by
>>>>>>>>>> deleting /controller
>>>>>>>>>>
>>>>>>>>>> -Clark
>>>>>>>>>>
>>>>>>>>>> Sent from my iPhone
>>>>>>>>>>
>>>>>>>>>>> On May 17, 2015, at 10:14 AM, Chris Neal <cw...@gmail.com>
>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hi Clark,
>>>>>>>>>>>
>>>>>>>>>>> Thank you for your reply! I do see that znode under /admin:
>>>>>>>>>>>
>>>>>>>>>>> root>
>> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
>>>>>>>>>>> -server myhost.mydomain.com:2181 ls /admin
>>>>>>>>>>> Connecting to myhost.mydomain.com:2181
>>>>>>>>>>> <snip>
>>>>>>>>>>> WATCHER::
>>>>>>>>>>>
>>>>>>>>>>> WatchedEvent state:SyncConnected type:None path:null
>>>>>>>>>>> [reassign_partitions, delete_topics]
>>>>>>>>>>>
>>>>>>>>>>> I'm not sure what this tells me though :)
>>>>>>>>>>> Again, thanks for your time.
>>>>>>>>>>> Chris
>>>>>>>>>>>
>>>>>>>>>>>> On Sun, May 17, 2015 at 12:20 AM, Clark Haskins
>> <clark@kafka.guru
>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Does the partition reassignment znode exist under /admin in
>>>>>> zookeeper?
>>>>>>>>>>>>
>>>>>>>>>>>> -Clark
>>>>>>>>>>>>
>>>>>>>>>>>> Sent from my iPhone
>>>>>>>>>>>>
>>>>>>>>>>>>> On May 16, 2015, at 7:16 PM, Chris Neal <cw...@gmail.com>
>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Sorry for bumping my own thread. :S Just wanted to get it in
>>>>>> front
>>>>>>>> of
>>>>>>>>>>>> some
>>>>>>>>>>>>> eyes again!
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks for your time and help.
>>>>>>>>>>>>> Chris
>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Thu, May 14, 2015 at 10:24 AM, Chris Neal <
>> cwneal@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I am running: kafka_2.10-0.8.1.1, and when I run the
>>>>>>>>>>>>>> reassign-partitions.sh script, I get this:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Partitions reassignment failed due to Partition reassignment
>>>>>>>> currently
>>>>>>>>>>>> in
>>>>>>>>>>>>>> progress for Map(). Aborting operation
>>>>>>>>>>>>>> kafka.common.AdminCommandFailedException: Partition
>> reassignment
>>>>>>>>>>>> currently
>>>>>>>>>>>>>> in progress for Map(). Aborting operation
>>>>>>>>>>>>>> at
>> kafka.admin.ReassignPartitionsCommand.reassignPartitions(ReassignPartitionsCommand.scala:204)
>>>>>>>>>>>>>> at
>> kafka.admin.ReassignPartitionsCommand$.executeAssignment(ReassignPartitionsCommand.scala:124)
>>>>>>>>>>>>>> at
>> kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:49)
>>>>>>>>>>>>>> at
>> kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I'm running it as:
>>>>>>>>>>>>>> /opt/kafka/kafka/bin/kafka-reassign-partitions.sh --zookeeper
>>>>>>>>>> $ZOOKEEPER
>>>>>>>>>>>>>> --reassignment-json-file ./$1.json --execute
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> and my json file looks like:
>>>>>>>>>>>>>> {"version":1,
>>>>>>>>>>>>>> "partitions":[
>>>>>>>>>>>>>> {"topic":"perf_mytopic_raw", "partition":0,"replicas":[0,1]},
>>>>>>>>>>>>>> {"topic":"perf_mytopic_raw","partition":1,"replicas":[0,1]}]}
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The empty Map() looks suspicious, and I found another post
>> from
>>>>>>>> about
>>>>>>>>>> a
>>>>>>>>>>>>>> year ago where it was thought that this might be a bug? I
>>>>>> looked
>>>>>>>>>>>> through
>>>>>>>>>>>>>> JIRA and didn't see anything that looked like this issue was
>>>>>> created
>>>>>>>>>> or
>>>>>>>>>>>>>> fixed, so I wanted to ask again about it.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Any ideas why this might be failing? There is definitely not
>>>>>>>> another
>>>>>>>>>>>>>> reassignment in progress. :)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks very much for your time.
>>>>>>>>>>>>>> Chris
>>
Re: Partitions reassignment failed due to Partition reassignment
currently in progress for Map()
Posted by Chris Neal <cw...@gmail.com>.
Done, and things look clean in Zookeeper now. I'm guessing that it is safe
for me to try to increase the replication count on this topic again? :)
Any idea what might have happened to put me in this state?
I had successfully increased probably 10 or so topics, but then I started
getting this message and nothing worked after that.
Again, thank you so much for your time and knowledge. Very much
appreciated.
Chris
On Sun, May 17, 2015 at 2:20 PM, Clark Haskins <cl...@kafka.guru> wrote:
> No problem.
>
> Delete the reassign_partitions znode. Then do the controller again.
>
> The controller znode gets instantly re-created but with a different broker
> as the controller.
>
> If you do a get /controller you can see the values change
>
> -Clark
>
> Sent from my iPhone
>
> > On May 17, 2015, at 11:48 AM, Chris Neal <cw...@gmail.com> wrote:
> >
> > Gah. Sorry for the spamming replies.
> >
> > The reassign_partitions znode is also still there, and still empty....For
> > what that's worth. :)
> > Back to thinking that deleting the reassign_partitions znode is the right
> > way to go.
> >
> > Chris
> >
> >> On Sun, May 17, 2015 at 1:46 PM, Chris Neal <cw...@gmail.com> wrote:
> >>
> >> Ok, I might have lied. Timezone got me. I do see this in the Kafka
> logs,
> >> so maybe it did work?
> >>
> >> [2015-05-17 14:36:57,924] INFO 0 successfully elected as leader
> >> (kafka.server.ZookeeperLeaderElector)
> >> [2015-05-17 14:36:57,924] INFO 0 successfully elected as leader
> >> (kafka.server.ZookeeperLeaderElector)
> >> [2015-05-17 14:36:58,975] INFO New leader is 0
> >> (kafka.server.ZookeeperLeaderElector$LeaderChangeListener)
> >> [2015-05-17 14:36:58,975] INFO New leader is 0
> >> (kafka.server.ZookeeperLeaderElector$LeaderChangeListener)
> >>
> >> Chris
> >>
> >>> On Sun, May 17, 2015 at 1:43 PM, Chris Neal <cw...@gmail.com> wrote:
> >>>
> >>> Thank you again. :)
> >>>
> >>> I ran the delete, but I'm not sure if it did anything:
> >>>
> >>> root>
> >>>
> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
> >>> -server myhost.mydomain.com:2181 delete /controller
> >>> Connecting to myhost.mydomain.com:2181
> >>>
> >>> WATCHER::
> >>>
> >>> WatchedEvent state:SyncConnected type:None path:null
> >>> root>
> >>>
> >>> I followed that up immediately by an "ls /", and the controller znode
> was
> >>> there. Either still there, or had already been re-created. There was
> >>> nothing in the Kafka logs indicating that anything had happened
> either, so
> >>> I'm thinking maybe I need to try and delete the znode partition
> manually?
> >>>
> >>> Or, should I have seen the controller znode disappear for a time?
> >>>
> >>> Very much appreciate your help!
> >>> Chris
> >>>
> >>>> On Sun, May 17, 2015 at 1:33 PM, Clark Haskins <cl...@kafka.guru>
> wrote:
> >>>>
> >>>> Yeah it's empty. So I would try and force a new controller election to
> >>>> see if that cleans up the znode.
> >>>>
> >>>> Delete /controller
> >>>>
> >>>> If that doesn't work you will have to delete the reassign_partitions
> >>>> znode manually
> >>>>
> >>>> -Clark
> >>>>
> >>>> Sent from my iPhone
> >>>>
> >>>>> On May 17, 2015, at 11:16 AM, Chris Neal <cw...@gmail.com> wrote:
> >>>>>
> >>>>> Sorry :)
> >>>>> Here you go:
> >>>>>
> >>>>> root>
> >>>>
> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
> >>>>> -server myhost.mydomain.com <http://myhost.mydomain.com:2181/>:2181
> >>>> get
> >>>>> /admin/reassign_partitions
> >>>>> Connecting to myhost.mydomain.com <http://myhost.mydomain.com:2181/
> >>>>> :2181
> >>>>> <snip>
> >>>>> WATCHER::
> >>>>>
> >>>>> WatchedEvent state:SyncConnected type:None path:null
> >>>>> {"version":1,"partitions":[]}
> >>>>> cZxid = 0x6001afde9
> >>>>> ctime = Wed Jan 21 18:37:40 UTC 2015
> >>>>> mZxid = 0x6001afde9
> >>>>> mtime = Wed Jan 21 18:37:40 UTC 2015
> >>>>> pZxid = 0x6001afde9
> >>>>> cversion = 0
> >>>>> dataVersion = 0
> >>>>> aclVersion = 0
> >>>>> ephemeralOwner = 0x0
> >>>>> dataLength = 29
> >>>>> numChildren = 0
> >>>>>
> >>>>> Thank you!
> >>>>> Chris
> >>>>>
> >>>>>> On Sun, May 17, 2015 at 12:58 PM, Clark Haskins <cl...@kafka.guru>
> >>>> wrote:
> >>>>>>
> >>>>>> Do a get /admin/reassign_partitions
> >>>>>>
> >>>>>> -Clark
> >>>>>>
> >>>>>> Sent from my iPhone
> >>>>>>
> >>>>>>> On May 17, 2015, at 10:20 AM, Chris Neal <cw...@gmail.com> wrote:
> >>>>>>>
> >>>>>>> Sure thing :)
> >>>>>>> Hopefully I did this right. Somewhat of a Zookeeper noob.
> >>>>>>>
> >>>>>>> root>
> >>>>
> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
> >>>>>>> -server myhost.mydomain.com <http://myhost.mydomain.com:2181/
> >:2181
> >>>> ls
> >>>>>>> /admin/reassign_partitions
> >>>>>>> Connecting to myhost.mydomain.com <
> http://myhost.mydomain.com:2181/
> >>>>>>> :2181
> >>>>>>> <snip>
> >>>>>>> WATCHER::
> >>>>>>>
> >>>>>>> WatchedEvent state:SyncConnected type:None path:null
> >>>>>>> []
> >>>>>>>
> >>>>>>> Hope that is helpful :)
> >>>>>>> If this is not what you were asking for, please just let me know.
> >>>>>>> Thank you!
> >>>>>>> Chris
> >>>>>>>
> >>>>>>>> On Sun, May 17, 2015 at 12:17 PM, Clark Haskins <clark@kafka.guru
> >
> >>>>>> wrote:
> >>>>>>>>
> >>>>>>>> The reassign_partitions znode is the important one. Please paste
> the
> >>>>>>>> contents of it. That node should only exist while there is a
> >>>>>> reassignment
> >>>>>>>> in progress.
> >>>>>>>>
> >>>>>>>> You can probably fix this up by forcing a new controller to come
> >>>> online
> >>>>>> by
> >>>>>>>> deleting /controller
> >>>>>>>>
> >>>>>>>> -Clark
> >>>>>>>>
> >>>>>>>> Sent from my iPhone
> >>>>>>>>
> >>>>>>>>> On May 17, 2015, at 10:14 AM, Chris Neal <cw...@gmail.com>
> wrote:
> >>>>>>>>>
> >>>>>>>>> Hi Clark,
> >>>>>>>>>
> >>>>>>>>> Thank you for your reply! I do see that znode under /admin:
> >>>>>>>>>
> >>>>>>>>> root>
> >>>>
> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
> >>>>>>>>> -server myhost.mydomain.com:2181 ls /admin
> >>>>>>>>> Connecting to myhost.mydomain.com:2181
> >>>>>>>>> <snip>
> >>>>>>>>> WATCHER::
> >>>>>>>>>
> >>>>>>>>> WatchedEvent state:SyncConnected type:None path:null
> >>>>>>>>> [reassign_partitions, delete_topics]
> >>>>>>>>>
> >>>>>>>>> I'm not sure what this tells me though :)
> >>>>>>>>> Again, thanks for your time.
> >>>>>>>>> Chris
> >>>>>>>>>
> >>>>>>>>>> On Sun, May 17, 2015 at 12:20 AM, Clark Haskins
> <clark@kafka.guru
> >>>>>
> >>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> Does the partition reassignment znode exist under /admin in
> >>>> zookeeper?
> >>>>>>>>>>
> >>>>>>>>>> -Clark
> >>>>>>>>>>
> >>>>>>>>>> Sent from my iPhone
> >>>>>>>>>>
> >>>>>>>>>>> On May 16, 2015, at 7:16 PM, Chris Neal <cw...@gmail.com>
> >>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> Sorry for bumping my own thread. :S Just wanted to get it in
> >>>> front
> >>>>>> of
> >>>>>>>>>> some
> >>>>>>>>>>> eyes again!
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks for your time and help.
> >>>>>>>>>>> Chris
> >>>>>>>>>>>
> >>>>>>>>>>>> On Thu, May 14, 2015 at 10:24 AM, Chris Neal <
> cwneal@gmail.com>
> >>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>
> >>>>>>>>>>>> I am running: kafka_2.10-0.8.1.1, and when I run the
> >>>>>>>>>>>> reassign-partitions.sh script, I get this:
> >>>>>>>>>>>>
> >>>>>>>>>>>> Partitions reassignment failed due to Partition reassignment
> >>>>>> currently
> >>>>>>>>>> in
> >>>>>>>>>>>> progress for Map(). Aborting operation
> >>>>>>>>>>>> kafka.common.AdminCommandFailedException: Partition
> reassignment
> >>>>>>>>>> currently
> >>>>>>>>>>>> in progress for Map(). Aborting operation
> >>>>>>>>>>>> at
> >>>>
> kafka.admin.ReassignPartitionsCommand.reassignPartitions(ReassignPartitionsCommand.scala:204)
> >>>>>>>>>>>> at
> >>>>
> kafka.admin.ReassignPartitionsCommand$.executeAssignment(ReassignPartitionsCommand.scala:124)
> >>>>>>>>>>>> at
> >>>>
> kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:49)
> >>>>>>>>>>>> at
> >>>>
> kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
> >>>>>>>>>>>>
> >>>>>>>>>>>> I'm running it as:
> >>>>>>>>>>>> /opt/kafka/kafka/bin/kafka-reassign-partitions.sh --zookeeper
> >>>>>>>> $ZOOKEEPER
> >>>>>>>>>>>> --reassignment-json-file ./$1.json --execute
> >>>>>>>>>>>>
> >>>>>>>>>>>> and my json file looks like:
> >>>>>>>>>>>> {"version":1,
> >>>>>>>>>>>> "partitions":[
> >>>>>>>>>>>> {"topic":"perf_mytopic_raw", "partition":0,"replicas":[0,1]},
> >>>>>>>>>>>> {"topic":"perf_mytopic_raw","partition":1,"replicas":[0,1]}]}
> >>>>>>>>>>>>
> >>>>>>>>>>>> The empty Map() looks suspicious, and I found another post
> from
> >>>>>> about
> >>>>>>>> a
> >>>>>>>>>>>> year ago where it was thought that this might be a bug? I
> >>>> looked
> >>>>>>>>>> through
> >>>>>>>>>>>> JIRA and didn't see anything that looked like this issue was
> >>>> created
> >>>>>>>> or
> >>>>>>>>>>>> fixed, so I wanted to ask again about it.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Any ideas why this might be failing? There is definitely not
> >>>>>> another
> >>>>>>>>>>>> reassignment in progress. :)
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks very much for your time.
> >>>>>>>>>>>> Chris
> >>
>
Re: Partitions reassignment failed due to Partition reassignment currently in progress for Map()
Posted by Clark Haskins <cl...@kafka.guru>.
No problem.
Delete the reassign_partitions znode. Then do the controller again.
The controller znode gets instantly re-created but with a different broker as the controller.
If you do a get /controller you can see the values change
-Clark
Sent from my iPhone
> On May 17, 2015, at 11:48 AM, Chris Neal <cw...@gmail.com> wrote:
>
> Gah. Sorry for the spamming replies.
>
> The reassign_partitions znode is also still there, and still empty....For
> what that's worth. :)
> Back to thinking that deleting the reassign_partitions znode is the right
> way to go.
>
> Chris
>
>> On Sun, May 17, 2015 at 1:46 PM, Chris Neal <cw...@gmail.com> wrote:
>>
>> Ok, I might have lied. Timezone got me. I do see this in the Kafka logs,
>> so maybe it did work?
>>
>> [2015-05-17 14:36:57,924] INFO 0 successfully elected as leader
>> (kafka.server.ZookeeperLeaderElector)
>> [2015-05-17 14:36:57,924] INFO 0 successfully elected as leader
>> (kafka.server.ZookeeperLeaderElector)
>> [2015-05-17 14:36:58,975] INFO New leader is 0
>> (kafka.server.ZookeeperLeaderElector$LeaderChangeListener)
>> [2015-05-17 14:36:58,975] INFO New leader is 0
>> (kafka.server.ZookeeperLeaderElector$LeaderChangeListener)
>>
>> Chris
>>
>>> On Sun, May 17, 2015 at 1:43 PM, Chris Neal <cw...@gmail.com> wrote:
>>>
>>> Thank you again. :)
>>>
>>> I ran the delete, but I'm not sure if it did anything:
>>>
>>> root>
>>> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
>>> -server myhost.mydomain.com:2181 delete /controller
>>> Connecting to myhost.mydomain.com:2181
>>>
>>> WATCHER::
>>>
>>> WatchedEvent state:SyncConnected type:None path:null
>>> root>
>>>
>>> I followed that up immediately by an "ls /", and the controller znode was
>>> there. Either still there, or had already been re-created. There was
>>> nothing in the Kafka logs indicating that anything had happened either, so
>>> I'm thinking maybe I need to try and delete the znode partition manually?
>>>
>>> Or, should I have seen the controller znode disappear for a time?
>>>
>>> Very much appreciate your help!
>>> Chris
>>>
>>>> On Sun, May 17, 2015 at 1:33 PM, Clark Haskins <cl...@kafka.guru> wrote:
>>>>
>>>> Yeah it's empty. So I would try and force a new controller election to
>>>> see if that cleans up the znode.
>>>>
>>>> Delete /controller
>>>>
>>>> If that doesn't work you will have to delete the reassign_partitions
>>>> znode manually
>>>>
>>>> -Clark
>>>>
>>>> Sent from my iPhone
>>>>
>>>>> On May 17, 2015, at 11:16 AM, Chris Neal <cw...@gmail.com> wrote:
>>>>>
>>>>> Sorry :)
>>>>> Here you go:
>>>>>
>>>>> root>
>>>> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
>>>>> -server myhost.mydomain.com <http://myhost.mydomain.com:2181/>:2181
>>>> get
>>>>> /admin/reassign_partitions
>>>>> Connecting to myhost.mydomain.com <http://myhost.mydomain.com:2181/
>>>>> :2181
>>>>> <snip>
>>>>> WATCHER::
>>>>>
>>>>> WatchedEvent state:SyncConnected type:None path:null
>>>>> {"version":1,"partitions":[]}
>>>>> cZxid = 0x6001afde9
>>>>> ctime = Wed Jan 21 18:37:40 UTC 2015
>>>>> mZxid = 0x6001afde9
>>>>> mtime = Wed Jan 21 18:37:40 UTC 2015
>>>>> pZxid = 0x6001afde9
>>>>> cversion = 0
>>>>> dataVersion = 0
>>>>> aclVersion = 0
>>>>> ephemeralOwner = 0x0
>>>>> dataLength = 29
>>>>> numChildren = 0
>>>>>
>>>>> Thank you!
>>>>> Chris
>>>>>
>>>>>> On Sun, May 17, 2015 at 12:58 PM, Clark Haskins <cl...@kafka.guru>
>>>> wrote:
>>>>>>
>>>>>> Do a get /admin/reassign_partitions
>>>>>>
>>>>>> -Clark
>>>>>>
>>>>>> Sent from my iPhone
>>>>>>
>>>>>>> On May 17, 2015, at 10:20 AM, Chris Neal <cw...@gmail.com> wrote:
>>>>>>>
>>>>>>> Sure thing :)
>>>>>>> Hopefully I did this right. Somewhat of a Zookeeper noob.
>>>>>>>
>>>>>>> root>
>>>> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
>>>>>>> -server myhost.mydomain.com <http://myhost.mydomain.com:2181/>:2181
>>>> ls
>>>>>>> /admin/reassign_partitions
>>>>>>> Connecting to myhost.mydomain.com <http://myhost.mydomain.com:2181/
>>>>>>> :2181
>>>>>>> <snip>
>>>>>>> WATCHER::
>>>>>>>
>>>>>>> WatchedEvent state:SyncConnected type:None path:null
>>>>>>> []
>>>>>>>
>>>>>>> Hope that is helpful :)
>>>>>>> If this is not what you were asking for, please just let me know.
>>>>>>> Thank you!
>>>>>>> Chris
>>>>>>>
>>>>>>>> On Sun, May 17, 2015 at 12:17 PM, Clark Haskins <cl...@kafka.guru>
>>>>>> wrote:
>>>>>>>>
>>>>>>>> The reassign_partitions znode is the important one. Please paste the
>>>>>>>> contents of it. That node should only exist while there is a
>>>>>> reassignment
>>>>>>>> in progress.
>>>>>>>>
>>>>>>>> You can probably fix this up by forcing a new controller to come
>>>> online
>>>>>> by
>>>>>>>> deleting /controller
>>>>>>>>
>>>>>>>> -Clark
>>>>>>>>
>>>>>>>> Sent from my iPhone
>>>>>>>>
>>>>>>>>> On May 17, 2015, at 10:14 AM, Chris Neal <cw...@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>> Hi Clark,
>>>>>>>>>
>>>>>>>>> Thank you for your reply! I do see that znode under /admin:
>>>>>>>>>
>>>>>>>>> root>
>>>> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
>>>>>>>>> -server myhost.mydomain.com:2181 ls /admin
>>>>>>>>> Connecting to myhost.mydomain.com:2181
>>>>>>>>> <snip>
>>>>>>>>> WATCHER::
>>>>>>>>>
>>>>>>>>> WatchedEvent state:SyncConnected type:None path:null
>>>>>>>>> [reassign_partitions, delete_topics]
>>>>>>>>>
>>>>>>>>> I'm not sure what this tells me though :)
>>>>>>>>> Again, thanks for your time.
>>>>>>>>> Chris
>>>>>>>>>
>>>>>>>>>> On Sun, May 17, 2015 at 12:20 AM, Clark Haskins <clark@kafka.guru
>>>>>
>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Does the partition reassignment znode exist under /admin in
>>>> zookeeper?
>>>>>>>>>>
>>>>>>>>>> -Clark
>>>>>>>>>>
>>>>>>>>>> Sent from my iPhone
>>>>>>>>>>
>>>>>>>>>>> On May 16, 2015, at 7:16 PM, Chris Neal <cw...@gmail.com>
>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Sorry for bumping my own thread. :S Just wanted to get it in
>>>> front
>>>>>> of
>>>>>>>>>> some
>>>>>>>>>>> eyes again!
>>>>>>>>>>>
>>>>>>>>>>> Thanks for your time and help.
>>>>>>>>>>> Chris
>>>>>>>>>>>
>>>>>>>>>>>> On Thu, May 14, 2015 at 10:24 AM, Chris Neal <cw...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>
>>>>>>>>>>>> I am running: kafka_2.10-0.8.1.1, and when I run the
>>>>>>>>>>>> reassign-partitions.sh script, I get this:
>>>>>>>>>>>>
>>>>>>>>>>>> Partitions reassignment failed due to Partition reassignment
>>>>>> currently
>>>>>>>>>> in
>>>>>>>>>>>> progress for Map(). Aborting operation
>>>>>>>>>>>> kafka.common.AdminCommandFailedException: Partition reassignment
>>>>>>>>>> currently
>>>>>>>>>>>> in progress for Map(). Aborting operation
>>>>>>>>>>>> at
>>>> kafka.admin.ReassignPartitionsCommand.reassignPartitions(ReassignPartitionsCommand.scala:204)
>>>>>>>>>>>> at
>>>> kafka.admin.ReassignPartitionsCommand$.executeAssignment(ReassignPartitionsCommand.scala:124)
>>>>>>>>>>>> at
>>>> kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:49)
>>>>>>>>>>>> at
>>>> kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
>>>>>>>>>>>>
>>>>>>>>>>>> I'm running it as:
>>>>>>>>>>>> /opt/kafka/kafka/bin/kafka-reassign-partitions.sh --zookeeper
>>>>>>>> $ZOOKEEPER
>>>>>>>>>>>> --reassignment-json-file ./$1.json --execute
>>>>>>>>>>>>
>>>>>>>>>>>> and my json file looks like:
>>>>>>>>>>>> {"version":1,
>>>>>>>>>>>> "partitions":[
>>>>>>>>>>>> {"topic":"perf_mytopic_raw", "partition":0,"replicas":[0,1]},
>>>>>>>>>>>> {"topic":"perf_mytopic_raw","partition":1,"replicas":[0,1]}]}
>>>>>>>>>>>>
>>>>>>>>>>>> The empty Map() looks suspicious, and I found another post from
>>>>>> about
>>>>>>>> a
>>>>>>>>>>>> year ago where it was thought that this might be a bug? I
>>>> looked
>>>>>>>>>> through
>>>>>>>>>>>> JIRA and didn't see anything that looked like this issue was
>>>> created
>>>>>>>> or
>>>>>>>>>>>> fixed, so I wanted to ask again about it.
>>>>>>>>>>>>
>>>>>>>>>>>> Any ideas why this might be failing? There is definitely not
>>>>>> another
>>>>>>>>>>>> reassignment in progress. :)
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks very much for your time.
>>>>>>>>>>>> Chris
>>
Re: Partitions reassignment failed due to Partition reassignment
currently in progress for Map()
Posted by Chris Neal <cw...@gmail.com>.
Gah. Sorry for the spamming replies.
The reassign_partitions znode is also still there, and still empty....For
what that's worth. :)
Back to thinking that deleting the reassign_partitions znode is the right
way to go.
Chris
On Sun, May 17, 2015 at 1:46 PM, Chris Neal <cw...@gmail.com> wrote:
> Ok, I might have lied. Timezone got me. I do see this in the Kafka logs,
> so maybe it did work?
>
> [2015-05-17 14:36:57,924] INFO 0 successfully elected as leader
> (kafka.server.ZookeeperLeaderElector)
> [2015-05-17 14:36:57,924] INFO 0 successfully elected as leader
> (kafka.server.ZookeeperLeaderElector)
> [2015-05-17 14:36:58,975] INFO New leader is 0
> (kafka.server.ZookeeperLeaderElector$LeaderChangeListener)
> [2015-05-17 14:36:58,975] INFO New leader is 0
> (kafka.server.ZookeeperLeaderElector$LeaderChangeListener)
>
> Chris
>
> On Sun, May 17, 2015 at 1:43 PM, Chris Neal <cw...@gmail.com> wrote:
>
>> Thank you again. :)
>>
>> I ran the delete, but I'm not sure if it did anything:
>>
>> root>
>> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
>> -server myhost.mydomain.com:2181 delete /controller
>> Connecting to myhost.mydomain.com:2181
>>
>> WATCHER::
>>
>> WatchedEvent state:SyncConnected type:None path:null
>> root>
>>
>> I followed that up immediately by an "ls /", and the controller znode was
>> there. Either still there, or had already been re-created. There was
>> nothing in the Kafka logs indicating that anything had happened either, so
>> I'm thinking maybe I need to try and delete the znode partition manually?
>>
>> Or, should I have seen the controller znode disappear for a time?
>>
>> Very much appreciate your help!
>> Chris
>>
>> On Sun, May 17, 2015 at 1:33 PM, Clark Haskins <cl...@kafka.guru> wrote:
>>
>>> Yeah it's empty. So I would try and force a new controller election to
>>> see if that cleans up the znode.
>>>
>>> Delete /controller
>>>
>>> If that doesn't work you will have to delete the reassign_partitions
>>> znode manually
>>>
>>> -Clark
>>>
>>> Sent from my iPhone
>>>
>>> > On May 17, 2015, at 11:16 AM, Chris Neal <cw...@gmail.com> wrote:
>>> >
>>> > Sorry :)
>>> > Here you go:
>>> >
>>> > root>
>>> >
>>> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
>>> > -server myhost.mydomain.com <http://myhost.mydomain.com:2181/>:2181
>>> get
>>> > /admin/reassign_partitions
>>> > Connecting to myhost.mydomain.com <http://myhost.mydomain.com:2181/
>>> >:2181
>>> > <snip>
>>> > WATCHER::
>>> >
>>> > WatchedEvent state:SyncConnected type:None path:null
>>> > {"version":1,"partitions":[]}
>>> > cZxid = 0x6001afde9
>>> > ctime = Wed Jan 21 18:37:40 UTC 2015
>>> > mZxid = 0x6001afde9
>>> > mtime = Wed Jan 21 18:37:40 UTC 2015
>>> > pZxid = 0x6001afde9
>>> > cversion = 0
>>> > dataVersion = 0
>>> > aclVersion = 0
>>> > ephemeralOwner = 0x0
>>> > dataLength = 29
>>> > numChildren = 0
>>> >
>>> > Thank you!
>>> > Chris
>>> >
>>> >> On Sun, May 17, 2015 at 12:58 PM, Clark Haskins <cl...@kafka.guru>
>>> wrote:
>>> >>
>>> >> Do a get /admin/reassign_partitions
>>> >>
>>> >> -Clark
>>> >>
>>> >> Sent from my iPhone
>>> >>
>>> >>> On May 17, 2015, at 10:20 AM, Chris Neal <cw...@gmail.com> wrote:
>>> >>>
>>> >>> Sure thing :)
>>> >>> Hopefully I did this right. Somewhat of a Zookeeper noob.
>>> >>>
>>> >>> root>
>>> >>
>>> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
>>> >>> -server myhost.mydomain.com <http://myhost.mydomain.com:2181/>:2181
>>> ls
>>> >>> /admin/reassign_partitions
>>> >>> Connecting to myhost.mydomain.com <http://myhost.mydomain.com:2181/
>>> >>> :2181
>>> >>> <snip>
>>> >>> WATCHER::
>>> >>>
>>> >>> WatchedEvent state:SyncConnected type:None path:null
>>> >>> []
>>> >>>
>>> >>> Hope that is helpful :)
>>> >>> If this is not what you were asking for, please just let me know.
>>> >>> Thank you!
>>> >>> Chris
>>> >>>
>>> >>>> On Sun, May 17, 2015 at 12:17 PM, Clark Haskins <cl...@kafka.guru>
>>> >> wrote:
>>> >>>>
>>> >>>> The reassign_partitions znode is the important one. Please paste the
>>> >>>> contents of it. That node should only exist while there is a
>>> >> reassignment
>>> >>>> in progress.
>>> >>>>
>>> >>>> You can probably fix this up by forcing a new controller to come
>>> online
>>> >> by
>>> >>>> deleting /controller
>>> >>>>
>>> >>>> -Clark
>>> >>>>
>>> >>>> Sent from my iPhone
>>> >>>>
>>> >>>>> On May 17, 2015, at 10:14 AM, Chris Neal <cw...@gmail.com> wrote:
>>> >>>>>
>>> >>>>> Hi Clark,
>>> >>>>>
>>> >>>>> Thank you for your reply! I do see that znode under /admin:
>>> >>>>>
>>> >>>>> root>
>>> >>
>>> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
>>> >>>>> -server myhost.mydomain.com:2181 ls /admin
>>> >>>>> Connecting to myhost.mydomain.com:2181
>>> >>>>> <snip>
>>> >>>>> WATCHER::
>>> >>>>>
>>> >>>>> WatchedEvent state:SyncConnected type:None path:null
>>> >>>>> [reassign_partitions, delete_topics]
>>> >>>>>
>>> >>>>> I'm not sure what this tells me though :)
>>> >>>>> Again, thanks for your time.
>>> >>>>> Chris
>>> >>>>>
>>> >>>>>> On Sun, May 17, 2015 at 12:20 AM, Clark Haskins <clark@kafka.guru
>>> >
>>> >>>> wrote:
>>> >>>>>>
>>> >>>>>> Does the partition reassignment znode exist under /admin in
>>> zookeeper?
>>> >>>>>>
>>> >>>>>> -Clark
>>> >>>>>>
>>> >>>>>> Sent from my iPhone
>>> >>>>>>
>>> >>>>>>> On May 16, 2015, at 7:16 PM, Chris Neal <cw...@gmail.com>
>>> wrote:
>>> >>>>>>>
>>> >>>>>>> Sorry for bumping my own thread. :S Just wanted to get it in
>>> front
>>> >> of
>>> >>>>>> some
>>> >>>>>>> eyes again!
>>> >>>>>>>
>>> >>>>>>> Thanks for your time and help.
>>> >>>>>>> Chris
>>> >>>>>>>
>>> >>>>>>>> On Thu, May 14, 2015 at 10:24 AM, Chris Neal <cw...@gmail.com>
>>> >>>> wrote:
>>> >>>>>>>>
>>> >>>>>>>> Hi All,
>>> >>>>>>>>
>>> >>>>>>>> I am running: kafka_2.10-0.8.1.1, and when I run the
>>> >>>>>>>> reassign-partitions.sh script, I get this:
>>> >>>>>>>>
>>> >>>>>>>> Partitions reassignment failed due to Partition reassignment
>>> >> currently
>>> >>>>>> in
>>> >>>>>>>> progress for Map(). Aborting operation
>>> >>>>>>>> kafka.common.AdminCommandFailedException: Partition reassignment
>>> >>>>>> currently
>>> >>>>>>>> in progress for Map(). Aborting operation
>>> >>>>>>>> at
>>> >>
>>> kafka.admin.ReassignPartitionsCommand.reassignPartitions(ReassignPartitionsCommand.scala:204)
>>> >>>>>>>> at
>>> >>
>>> kafka.admin.ReassignPartitionsCommand$.executeAssignment(ReassignPartitionsCommand.scala:124)
>>> >>>>>>>> at
>>> >>
>>> kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:49)
>>> >>>>>>>> at
>>> >>
>>> kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
>>> >>>>>>>>
>>> >>>>>>>> I'm running it as:
>>> >>>>>>>> /opt/kafka/kafka/bin/kafka-reassign-partitions.sh --zookeeper
>>> >>>> $ZOOKEEPER
>>> >>>>>>>> --reassignment-json-file ./$1.json --execute
>>> >>>>>>>>
>>> >>>>>>>> and my json file looks like:
>>> >>>>>>>> {"version":1,
>>> >>>>>>>> "partitions":[
>>> >>>>>>>> {"topic":"perf_mytopic_raw", "partition":0,"replicas":[0,1]},
>>> >>>>>>>> {"topic":"perf_mytopic_raw","partition":1,"replicas":[0,1]}]}
>>> >>>>>>>>
>>> >>>>>>>> The empty Map() looks suspicious, and I found another post from
>>> >> about
>>> >>>> a
>>> >>>>>>>> year ago where it was thought that this might be a bug? I
>>> looked
>>> >>>>>> through
>>> >>>>>>>> JIRA and didn't see anything that looked like this issue was
>>> created
>>> >>>> or
>>> >>>>>>>> fixed, so I wanted to ask again about it.
>>> >>>>>>>>
>>> >>>>>>>> Any ideas why this might be failing? There is definitely not
>>> >> another
>>> >>>>>>>> reassignment in progress. :)
>>> >>>>>>>>
>>> >>>>>>>> Thanks very much for your time.
>>> >>>>>>>> Chris
>>> >>
>>>
>>
>>
>
Re: Partitions reassignment failed due to Partition reassignment
currently in progress for Map()
Posted by Chris Neal <cw...@gmail.com>.
Ok, I might have lied. Timezone got me. I do see this in the Kafka logs,
so maybe it did work?
[2015-05-17 14:36:57,924] INFO 0 successfully elected as leader
(kafka.server.ZookeeperLeaderElector)
[2015-05-17 14:36:57,924] INFO 0 successfully elected as leader
(kafka.server.ZookeeperLeaderElector)
[2015-05-17 14:36:58,975] INFO New leader is 0
(kafka.server.ZookeeperLeaderElector$LeaderChangeListener)
[2015-05-17 14:36:58,975] INFO New leader is 0
(kafka.server.ZookeeperLeaderElector$LeaderChangeListener)
Chris
On Sun, May 17, 2015 at 1:43 PM, Chris Neal <cw...@gmail.com> wrote:
> Thank you again. :)
>
> I ran the delete, but I'm not sure if it did anything:
>
> root>
> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
> -server myhost.mydomain.com:2181 delete /controller
> Connecting to myhost.mydomain.com:2181
>
> WATCHER::
>
> WatchedEvent state:SyncConnected type:None path:null
> root>
>
> I followed that up immediately by an "ls /", and the controller znode was
> there. Either still there, or had already been re-created. There was
> nothing in the Kafka logs indicating that anything had happened either, so
> I'm thinking maybe I need to try and delete the znode partition manually?
>
> Or, should I have seen the controller znode disappear for a time?
>
> Very much appreciate your help!
> Chris
>
> On Sun, May 17, 2015 at 1:33 PM, Clark Haskins <cl...@kafka.guru> wrote:
>
>> Yeah it's empty. So I would try and force a new controller election to
>> see if that cleans up the znode.
>>
>> Delete /controller
>>
>> If that doesn't work you will have to delete the reassign_partitions
>> znode manually
>>
>> -Clark
>>
>> Sent from my iPhone
>>
>> > On May 17, 2015, at 11:16 AM, Chris Neal <cw...@gmail.com> wrote:
>> >
>> > Sorry :)
>> > Here you go:
>> >
>> > root>
>> >
>> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
>> > -server myhost.mydomain.com <http://myhost.mydomain.com:2181/>:2181 get
>> > /admin/reassign_partitions
>> > Connecting to myhost.mydomain.com <http://myhost.mydomain.com:2181/
>> >:2181
>> > <snip>
>> > WATCHER::
>> >
>> > WatchedEvent state:SyncConnected type:None path:null
>> > {"version":1,"partitions":[]}
>> > cZxid = 0x6001afde9
>> > ctime = Wed Jan 21 18:37:40 UTC 2015
>> > mZxid = 0x6001afde9
>> > mtime = Wed Jan 21 18:37:40 UTC 2015
>> > pZxid = 0x6001afde9
>> > cversion = 0
>> > dataVersion = 0
>> > aclVersion = 0
>> > ephemeralOwner = 0x0
>> > dataLength = 29
>> > numChildren = 0
>> >
>> > Thank you!
>> > Chris
>> >
>> >> On Sun, May 17, 2015 at 12:58 PM, Clark Haskins <cl...@kafka.guru>
>> wrote:
>> >>
>> >> Do a get /admin/reassign_partitions
>> >>
>> >> -Clark
>> >>
>> >> Sent from my iPhone
>> >>
>> >>> On May 17, 2015, at 10:20 AM, Chris Neal <cw...@gmail.com> wrote:
>> >>>
>> >>> Sure thing :)
>> >>> Hopefully I did this right. Somewhat of a Zookeeper noob.
>> >>>
>> >>> root>
>> >>
>> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
>> >>> -server myhost.mydomain.com <http://myhost.mydomain.com:2181/>:2181
>> ls
>> >>> /admin/reassign_partitions
>> >>> Connecting to myhost.mydomain.com <http://myhost.mydomain.com:2181/
>> >>> :2181
>> >>> <snip>
>> >>> WATCHER::
>> >>>
>> >>> WatchedEvent state:SyncConnected type:None path:null
>> >>> []
>> >>>
>> >>> Hope that is helpful :)
>> >>> If this is not what you were asking for, please just let me know.
>> >>> Thank you!
>> >>> Chris
>> >>>
>> >>>> On Sun, May 17, 2015 at 12:17 PM, Clark Haskins <cl...@kafka.guru>
>> >> wrote:
>> >>>>
>> >>>> The reassign_partitions znode is the important one. Please paste the
>> >>>> contents of it. That node should only exist while there is a
>> >> reassignment
>> >>>> in progress.
>> >>>>
>> >>>> You can probably fix this up by forcing a new controller to come
>> online
>> >> by
>> >>>> deleting /controller
>> >>>>
>> >>>> -Clark
>> >>>>
>> >>>> Sent from my iPhone
>> >>>>
>> >>>>> On May 17, 2015, at 10:14 AM, Chris Neal <cw...@gmail.com> wrote:
>> >>>>>
>> >>>>> Hi Clark,
>> >>>>>
>> >>>>> Thank you for your reply! I do see that znode under /admin:
>> >>>>>
>> >>>>> root>
>> >>
>> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
>> >>>>> -server myhost.mydomain.com:2181 ls /admin
>> >>>>> Connecting to myhost.mydomain.com:2181
>> >>>>> <snip>
>> >>>>> WATCHER::
>> >>>>>
>> >>>>> WatchedEvent state:SyncConnected type:None path:null
>> >>>>> [reassign_partitions, delete_topics]
>> >>>>>
>> >>>>> I'm not sure what this tells me though :)
>> >>>>> Again, thanks for your time.
>> >>>>> Chris
>> >>>>>
>> >>>>>> On Sun, May 17, 2015 at 12:20 AM, Clark Haskins <cl...@kafka.guru>
>> >>>> wrote:
>> >>>>>>
>> >>>>>> Does the partition reassignment znode exist under /admin in
>> zookeeper?
>> >>>>>>
>> >>>>>> -Clark
>> >>>>>>
>> >>>>>> Sent from my iPhone
>> >>>>>>
>> >>>>>>> On May 16, 2015, at 7:16 PM, Chris Neal <cw...@gmail.com> wrote:
>> >>>>>>>
>> >>>>>>> Sorry for bumping my own thread. :S Just wanted to get it in
>> front
>> >> of
>> >>>>>> some
>> >>>>>>> eyes again!
>> >>>>>>>
>> >>>>>>> Thanks for your time and help.
>> >>>>>>> Chris
>> >>>>>>>
>> >>>>>>>> On Thu, May 14, 2015 at 10:24 AM, Chris Neal <cw...@gmail.com>
>> >>>> wrote:
>> >>>>>>>>
>> >>>>>>>> Hi All,
>> >>>>>>>>
>> >>>>>>>> I am running: kafka_2.10-0.8.1.1, and when I run the
>> >>>>>>>> reassign-partitions.sh script, I get this:
>> >>>>>>>>
>> >>>>>>>> Partitions reassignment failed due to Partition reassignment
>> >> currently
>> >>>>>> in
>> >>>>>>>> progress for Map(). Aborting operation
>> >>>>>>>> kafka.common.AdminCommandFailedException: Partition reassignment
>> >>>>>> currently
>> >>>>>>>> in progress for Map(). Aborting operation
>> >>>>>>>> at
>> >>
>> kafka.admin.ReassignPartitionsCommand.reassignPartitions(ReassignPartitionsCommand.scala:204)
>> >>>>>>>> at
>> >>
>> kafka.admin.ReassignPartitionsCommand$.executeAssignment(ReassignPartitionsCommand.scala:124)
>> >>>>>>>> at
>> >>
>> kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:49)
>> >>>>>>>> at
>> >>
>> kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
>> >>>>>>>>
>> >>>>>>>> I'm running it as:
>> >>>>>>>> /opt/kafka/kafka/bin/kafka-reassign-partitions.sh --zookeeper
>> >>>> $ZOOKEEPER
>> >>>>>>>> --reassignment-json-file ./$1.json --execute
>> >>>>>>>>
>> >>>>>>>> and my json file looks like:
>> >>>>>>>> {"version":1,
>> >>>>>>>> "partitions":[
>> >>>>>>>> {"topic":"perf_mytopic_raw", "partition":0,"replicas":[0,1]},
>> >>>>>>>> {"topic":"perf_mytopic_raw","partition":1,"replicas":[0,1]}]}
>> >>>>>>>>
>> >>>>>>>> The empty Map() looks suspicious, and I found another post from
>> >> about
>> >>>> a
>> >>>>>>>> year ago where it was thought that this might be a bug? I looked
>> >>>>>> through
>> >>>>>>>> JIRA and didn't see anything that looked like this issue was
>> created
>> >>>> or
>> >>>>>>>> fixed, so I wanted to ask again about it.
>> >>>>>>>>
>> >>>>>>>> Any ideas why this might be failing? There is definitely not
>> >> another
>> >>>>>>>> reassignment in progress. :)
>> >>>>>>>>
>> >>>>>>>> Thanks very much for your time.
>> >>>>>>>> Chris
>> >>
>>
>
>
Re: Partitions reassignment failed due to Partition reassignment
currently in progress for Map()
Posted by Chris Neal <cw...@gmail.com>.
Thank you again. :)
I ran the delete, but I'm not sure if it did anything:
root>
./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
-server myhost.mydomain.com:2181 delete /controller
Connecting to myhost.mydomain.com:2181
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
root>
I followed that up immediately by an "ls /", and the controller znode was
there. Either still there, or had already been re-created. There was
nothing in the Kafka logs indicating that anything had happened either, so
I'm thinking maybe I need to try and delete the znode partition manually?
Or, should I have seen the controller znode disappear for a time?
Very much appreciate your help!
Chris
On Sun, May 17, 2015 at 1:33 PM, Clark Haskins <cl...@kafka.guru> wrote:
> Yeah it's empty. So I would try and force a new controller election to see
> if that cleans up the znode.
>
> Delete /controller
>
> If that doesn't work you will have to delete the reassign_partitions znode
> manually
>
> -Clark
>
> Sent from my iPhone
>
> > On May 17, 2015, at 11:16 AM, Chris Neal <cw...@gmail.com> wrote:
> >
> > Sorry :)
> > Here you go:
> >
> > root>
> >
> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
> > -server myhost.mydomain.com <http://myhost.mydomain.com:2181/>:2181 get
> > /admin/reassign_partitions
> > Connecting to myhost.mydomain.com <http://myhost.mydomain.com:2181/
> >:2181
> > <snip>
> > WATCHER::
> >
> > WatchedEvent state:SyncConnected type:None path:null
> > {"version":1,"partitions":[]}
> > cZxid = 0x6001afde9
> > ctime = Wed Jan 21 18:37:40 UTC 2015
> > mZxid = 0x6001afde9
> > mtime = Wed Jan 21 18:37:40 UTC 2015
> > pZxid = 0x6001afde9
> > cversion = 0
> > dataVersion = 0
> > aclVersion = 0
> > ephemeralOwner = 0x0
> > dataLength = 29
> > numChildren = 0
> >
> > Thank you!
> > Chris
> >
> >> On Sun, May 17, 2015 at 12:58 PM, Clark Haskins <cl...@kafka.guru>
> wrote:
> >>
> >> Do a get /admin/reassign_partitions
> >>
> >> -Clark
> >>
> >> Sent from my iPhone
> >>
> >>> On May 17, 2015, at 10:20 AM, Chris Neal <cw...@gmail.com> wrote:
> >>>
> >>> Sure thing :)
> >>> Hopefully I did this right. Somewhat of a Zookeeper noob.
> >>>
> >>> root>
> >>
> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
> >>> -server myhost.mydomain.com <http://myhost.mydomain.com:2181/>:2181 ls
> >>> /admin/reassign_partitions
> >>> Connecting to myhost.mydomain.com <http://myhost.mydomain.com:2181/
> >>> :2181
> >>> <snip>
> >>> WATCHER::
> >>>
> >>> WatchedEvent state:SyncConnected type:None path:null
> >>> []
> >>>
> >>> Hope that is helpful :)
> >>> If this is not what you were asking for, please just let me know.
> >>> Thank you!
> >>> Chris
> >>>
> >>>> On Sun, May 17, 2015 at 12:17 PM, Clark Haskins <cl...@kafka.guru>
> >> wrote:
> >>>>
> >>>> The reassign_partitions znode is the important one. Please paste the
> >>>> contents of it. That node should only exist while there is a
> >> reassignment
> >>>> in progress.
> >>>>
> >>>> You can probably fix this up by forcing a new controller to come
> online
> >> by
> >>>> deleting /controller
> >>>>
> >>>> -Clark
> >>>>
> >>>> Sent from my iPhone
> >>>>
> >>>>> On May 17, 2015, at 10:14 AM, Chris Neal <cw...@gmail.com> wrote:
> >>>>>
> >>>>> Hi Clark,
> >>>>>
> >>>>> Thank you for your reply! I do see that znode under /admin:
> >>>>>
> >>>>> root>
> >>
> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
> >>>>> -server myhost.mydomain.com:2181 ls /admin
> >>>>> Connecting to myhost.mydomain.com:2181
> >>>>> <snip>
> >>>>> WATCHER::
> >>>>>
> >>>>> WatchedEvent state:SyncConnected type:None path:null
> >>>>> [reassign_partitions, delete_topics]
> >>>>>
> >>>>> I'm not sure what this tells me though :)
> >>>>> Again, thanks for your time.
> >>>>> Chris
> >>>>>
> >>>>>> On Sun, May 17, 2015 at 12:20 AM, Clark Haskins <cl...@kafka.guru>
> >>>> wrote:
> >>>>>>
> >>>>>> Does the partition reassignment znode exist under /admin in
> zookeeper?
> >>>>>>
> >>>>>> -Clark
> >>>>>>
> >>>>>> Sent from my iPhone
> >>>>>>
> >>>>>>> On May 16, 2015, at 7:16 PM, Chris Neal <cw...@gmail.com> wrote:
> >>>>>>>
> >>>>>>> Sorry for bumping my own thread. :S Just wanted to get it in front
> >> of
> >>>>>> some
> >>>>>>> eyes again!
> >>>>>>>
> >>>>>>> Thanks for your time and help.
> >>>>>>> Chris
> >>>>>>>
> >>>>>>>> On Thu, May 14, 2015 at 10:24 AM, Chris Neal <cw...@gmail.com>
> >>>> wrote:
> >>>>>>>>
> >>>>>>>> Hi All,
> >>>>>>>>
> >>>>>>>> I am running: kafka_2.10-0.8.1.1, and when I run the
> >>>>>>>> reassign-partitions.sh script, I get this:
> >>>>>>>>
> >>>>>>>> Partitions reassignment failed due to Partition reassignment
> >> currently
> >>>>>> in
> >>>>>>>> progress for Map(). Aborting operation
> >>>>>>>> kafka.common.AdminCommandFailedException: Partition reassignment
> >>>>>> currently
> >>>>>>>> in progress for Map(). Aborting operation
> >>>>>>>> at
> >>
> kafka.admin.ReassignPartitionsCommand.reassignPartitions(ReassignPartitionsCommand.scala:204)
> >>>>>>>> at
> >>
> kafka.admin.ReassignPartitionsCommand$.executeAssignment(ReassignPartitionsCommand.scala:124)
> >>>>>>>> at
> >>
> kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:49)
> >>>>>>>> at
> >>
> kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
> >>>>>>>>
> >>>>>>>> I'm running it as:
> >>>>>>>> /opt/kafka/kafka/bin/kafka-reassign-partitions.sh --zookeeper
> >>>> $ZOOKEEPER
> >>>>>>>> --reassignment-json-file ./$1.json --execute
> >>>>>>>>
> >>>>>>>> and my json file looks like:
> >>>>>>>> {"version":1,
> >>>>>>>> "partitions":[
> >>>>>>>> {"topic":"perf_mytopic_raw", "partition":0,"replicas":[0,1]},
> >>>>>>>> {"topic":"perf_mytopic_raw","partition":1,"replicas":[0,1]}]}
> >>>>>>>>
> >>>>>>>> The empty Map() looks suspicious, and I found another post from
> >> about
> >>>> a
> >>>>>>>> year ago where it was thought that this might be a bug? I looked
> >>>>>> through
> >>>>>>>> JIRA and didn't see anything that looked like this issue was
> created
> >>>> or
> >>>>>>>> fixed, so I wanted to ask again about it.
> >>>>>>>>
> >>>>>>>> Any ideas why this might be failing? There is definitely not
> >> another
> >>>>>>>> reassignment in progress. :)
> >>>>>>>>
> >>>>>>>> Thanks very much for your time.
> >>>>>>>> Chris
> >>
>
Re: Partitions reassignment failed due to Partition reassignment currently in progress for Map()
Posted by Clark Haskins <cl...@kafka.guru>.
Yeah it's empty. So I would try and force a new controller election to see if that cleans up the znode.
Delete /controller
If that doesn't work you will have to delete the reassign_partitions znode manually
-Clark
Sent from my iPhone
> On May 17, 2015, at 11:16 AM, Chris Neal <cw...@gmail.com> wrote:
>
> Sorry :)
> Here you go:
>
> root>
> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
> -server myhost.mydomain.com <http://myhost.mydomain.com:2181/>:2181 get
> /admin/reassign_partitions
> Connecting to myhost.mydomain.com <http://myhost.mydomain.com:2181/>:2181
> <snip>
> WATCHER::
>
> WatchedEvent state:SyncConnected type:None path:null
> {"version":1,"partitions":[]}
> cZxid = 0x6001afde9
> ctime = Wed Jan 21 18:37:40 UTC 2015
> mZxid = 0x6001afde9
> mtime = Wed Jan 21 18:37:40 UTC 2015
> pZxid = 0x6001afde9
> cversion = 0
> dataVersion = 0
> aclVersion = 0
> ephemeralOwner = 0x0
> dataLength = 29
> numChildren = 0
>
> Thank you!
> Chris
>
>> On Sun, May 17, 2015 at 12:58 PM, Clark Haskins <cl...@kafka.guru> wrote:
>>
>> Do a get /admin/reassign_partitions
>>
>> -Clark
>>
>> Sent from my iPhone
>>
>>> On May 17, 2015, at 10:20 AM, Chris Neal <cw...@gmail.com> wrote:
>>>
>>> Sure thing :)
>>> Hopefully I did this right. Somewhat of a Zookeeper noob.
>>>
>>> root>
>> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
>>> -server myhost.mydomain.com <http://myhost.mydomain.com:2181/>:2181 ls
>>> /admin/reassign_partitions
>>> Connecting to myhost.mydomain.com <http://myhost.mydomain.com:2181/
>>> :2181
>>> <snip>
>>> WATCHER::
>>>
>>> WatchedEvent state:SyncConnected type:None path:null
>>> []
>>>
>>> Hope that is helpful :)
>>> If this is not what you were asking for, please just let me know.
>>> Thank you!
>>> Chris
>>>
>>>> On Sun, May 17, 2015 at 12:17 PM, Clark Haskins <cl...@kafka.guru>
>> wrote:
>>>>
>>>> The reassign_partitions znode is the important one. Please paste the
>>>> contents of it. That node should only exist while there is a
>> reassignment
>>>> in progress.
>>>>
>>>> You can probably fix this up by forcing a new controller to come online
>> by
>>>> deleting /controller
>>>>
>>>> -Clark
>>>>
>>>> Sent from my iPhone
>>>>
>>>>> On May 17, 2015, at 10:14 AM, Chris Neal <cw...@gmail.com> wrote:
>>>>>
>>>>> Hi Clark,
>>>>>
>>>>> Thank you for your reply! I do see that znode under /admin:
>>>>>
>>>>> root>
>> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
>>>>> -server myhost.mydomain.com:2181 ls /admin
>>>>> Connecting to myhost.mydomain.com:2181
>>>>> <snip>
>>>>> WATCHER::
>>>>>
>>>>> WatchedEvent state:SyncConnected type:None path:null
>>>>> [reassign_partitions, delete_topics]
>>>>>
>>>>> I'm not sure what this tells me though :)
>>>>> Again, thanks for your time.
>>>>> Chris
>>>>>
>>>>>> On Sun, May 17, 2015 at 12:20 AM, Clark Haskins <cl...@kafka.guru>
>>>> wrote:
>>>>>>
>>>>>> Does the partition reassignment znode exist under /admin in zookeeper?
>>>>>>
>>>>>> -Clark
>>>>>>
>>>>>> Sent from my iPhone
>>>>>>
>>>>>>> On May 16, 2015, at 7:16 PM, Chris Neal <cw...@gmail.com> wrote:
>>>>>>>
>>>>>>> Sorry for bumping my own thread. :S Just wanted to get it in front
>> of
>>>>>> some
>>>>>>> eyes again!
>>>>>>>
>>>>>>> Thanks for your time and help.
>>>>>>> Chris
>>>>>>>
>>>>>>>> On Thu, May 14, 2015 at 10:24 AM, Chris Neal <cw...@gmail.com>
>>>> wrote:
>>>>>>>>
>>>>>>>> Hi All,
>>>>>>>>
>>>>>>>> I am running: kafka_2.10-0.8.1.1, and when I run the
>>>>>>>> reassign-partitions.sh script, I get this:
>>>>>>>>
>>>>>>>> Partitions reassignment failed due to Partition reassignment
>> currently
>>>>>> in
>>>>>>>> progress for Map(). Aborting operation
>>>>>>>> kafka.common.AdminCommandFailedException: Partition reassignment
>>>>>> currently
>>>>>>>> in progress for Map(). Aborting operation
>>>>>>>> at
>> kafka.admin.ReassignPartitionsCommand.reassignPartitions(ReassignPartitionsCommand.scala:204)
>>>>>>>> at
>> kafka.admin.ReassignPartitionsCommand$.executeAssignment(ReassignPartitionsCommand.scala:124)
>>>>>>>> at
>> kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:49)
>>>>>>>> at
>> kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
>>>>>>>>
>>>>>>>> I'm running it as:
>>>>>>>> /opt/kafka/kafka/bin/kafka-reassign-partitions.sh --zookeeper
>>>> $ZOOKEEPER
>>>>>>>> --reassignment-json-file ./$1.json --execute
>>>>>>>>
>>>>>>>> and my json file looks like:
>>>>>>>> {"version":1,
>>>>>>>> "partitions":[
>>>>>>>> {"topic":"perf_mytopic_raw", "partition":0,"replicas":[0,1]},
>>>>>>>> {"topic":"perf_mytopic_raw","partition":1,"replicas":[0,1]}]}
>>>>>>>>
>>>>>>>> The empty Map() looks suspicious, and I found another post from
>> about
>>>> a
>>>>>>>> year ago where it was thought that this might be a bug? I looked
>>>>>> through
>>>>>>>> JIRA and didn't see anything that looked like this issue was created
>>>> or
>>>>>>>> fixed, so I wanted to ask again about it.
>>>>>>>>
>>>>>>>> Any ideas why this might be failing? There is definitely not
>> another
>>>>>>>> reassignment in progress. :)
>>>>>>>>
>>>>>>>> Thanks very much for your time.
>>>>>>>> Chris
>>
Re: Partitions reassignment failed due to Partition reassignment
currently in progress for Map()
Posted by Chris Neal <cw...@gmail.com>.
Sorry :)
Here you go:
root>
./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
-server myhost.mydomain.com <http://myhost.mydomain.com:2181/>:2181 get
/admin/reassign_partitions
Connecting to myhost.mydomain.com <http://myhost.mydomain.com:2181/>:2181
<snip>
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
{"version":1,"partitions":[]}
cZxid = 0x6001afde9
ctime = Wed Jan 21 18:37:40 UTC 2015
mZxid = 0x6001afde9
mtime = Wed Jan 21 18:37:40 UTC 2015
pZxid = 0x6001afde9
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 29
numChildren = 0
Thank you!
Chris
On Sun, May 17, 2015 at 12:58 PM, Clark Haskins <cl...@kafka.guru> wrote:
> Do a get /admin/reassign_partitions
>
> -Clark
>
> Sent from my iPhone
>
> > On May 17, 2015, at 10:20 AM, Chris Neal <cw...@gmail.com> wrote:
> >
> > Sure thing :)
> > Hopefully I did this right. Somewhat of a Zookeeper noob.
> >
> > root>
> >
> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
> > -server myhost.mydomain.com <http://myhost.mydomain.com:2181/>:2181 ls
> > /admin/reassign_partitions
> > Connecting to myhost.mydomain.com <http://myhost.mydomain.com:2181/
> >:2181
> > <snip>
> > WATCHER::
> >
> > WatchedEvent state:SyncConnected type:None path:null
> > []
> >
> > Hope that is helpful :)
> > If this is not what you were asking for, please just let me know.
> > Thank you!
> > Chris
> >
> >> On Sun, May 17, 2015 at 12:17 PM, Clark Haskins <cl...@kafka.guru>
> wrote:
> >>
> >> The reassign_partitions znode is the important one. Please paste the
> >> contents of it. That node should only exist while there is a
> reassignment
> >> in progress.
> >>
> >> You can probably fix this up by forcing a new controller to come online
> by
> >> deleting /controller
> >>
> >> -Clark
> >>
> >> Sent from my iPhone
> >>
> >>> On May 17, 2015, at 10:14 AM, Chris Neal <cw...@gmail.com> wrote:
> >>>
> >>> Hi Clark,
> >>>
> >>> Thank you for your reply! I do see that znode under /admin:
> >>>
> >>> root>
> >>
> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
> >>> -server myhost.mydomain.com:2181 ls /admin
> >>> Connecting to myhost.mydomain.com:2181
> >>> <snip>
> >>> WATCHER::
> >>>
> >>> WatchedEvent state:SyncConnected type:None path:null
> >>> [reassign_partitions, delete_topics]
> >>>
> >>> I'm not sure what this tells me though :)
> >>> Again, thanks for your time.
> >>> Chris
> >>>
> >>>> On Sun, May 17, 2015 at 12:20 AM, Clark Haskins <cl...@kafka.guru>
> >> wrote:
> >>>>
> >>>> Does the partition reassignment znode exist under /admin in zookeeper?
> >>>>
> >>>> -Clark
> >>>>
> >>>> Sent from my iPhone
> >>>>
> >>>>> On May 16, 2015, at 7:16 PM, Chris Neal <cw...@gmail.com> wrote:
> >>>>>
> >>>>> Sorry for bumping my own thread. :S Just wanted to get it in front
> of
> >>>> some
> >>>>> eyes again!
> >>>>>
> >>>>> Thanks for your time and help.
> >>>>> Chris
> >>>>>
> >>>>>> On Thu, May 14, 2015 at 10:24 AM, Chris Neal <cw...@gmail.com>
> >> wrote:
> >>>>>>
> >>>>>> Hi All,
> >>>>>>
> >>>>>> I am running: kafka_2.10-0.8.1.1, and when I run the
> >>>>>> reassign-partitions.sh script, I get this:
> >>>>>>
> >>>>>> Partitions reassignment failed due to Partition reassignment
> currently
> >>>> in
> >>>>>> progress for Map(). Aborting operation
> >>>>>> kafka.common.AdminCommandFailedException: Partition reassignment
> >>>> currently
> >>>>>> in progress for Map(). Aborting operation
> >>>>>> at
> >>
> kafka.admin.ReassignPartitionsCommand.reassignPartitions(ReassignPartitionsCommand.scala:204)
> >>>>>> at
> >>
> kafka.admin.ReassignPartitionsCommand$.executeAssignment(ReassignPartitionsCommand.scala:124)
> >>>>>> at
> >>
> kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:49)
> >>>>>> at
> >>
> kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
> >>>>>>
> >>>>>> I'm running it as:
> >>>>>> /opt/kafka/kafka/bin/kafka-reassign-partitions.sh --zookeeper
> >> $ZOOKEEPER
> >>>>>> --reassignment-json-file ./$1.json --execute
> >>>>>>
> >>>>>> and my json file looks like:
> >>>>>> {"version":1,
> >>>>>> "partitions":[
> >>>>>> {"topic":"perf_mytopic_raw", "partition":0,"replicas":[0,1]},
> >>>>>> {"topic":"perf_mytopic_raw","partition":1,"replicas":[0,1]}]}
> >>>>>>
> >>>>>> The empty Map() looks suspicious, and I found another post from
> about
> >> a
> >>>>>> year ago where it was thought that this might be a bug? I looked
> >>>> through
> >>>>>> JIRA and didn't see anything that looked like this issue was created
> >> or
> >>>>>> fixed, so I wanted to ask again about it.
> >>>>>>
> >>>>>> Any ideas why this might be failing? There is definitely not
> another
> >>>>>> reassignment in progress. :)
> >>>>>>
> >>>>>> Thanks very much for your time.
> >>>>>> Chris
> >>
>
Re: Partitions reassignment failed due to Partition reassignment currently in progress for Map()
Posted by Clark Haskins <cl...@kafka.guru>.
Do a get /admin/reassign_partitions
-Clark
Sent from my iPhone
> On May 17, 2015, at 10:20 AM, Chris Neal <cw...@gmail.com> wrote:
>
> Sure thing :)
> Hopefully I did this right. Somewhat of a Zookeeper noob.
>
> root>
> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
> -server myhost.mydomain.com <http://myhost.mydomain.com:2181/>:2181 ls
> /admin/reassign_partitions
> Connecting to myhost.mydomain.com <http://myhost.mydomain.com:2181/>:2181
> <snip>
> WATCHER::
>
> WatchedEvent state:SyncConnected type:None path:null
> []
>
> Hope that is helpful :)
> If this is not what you were asking for, please just let me know.
> Thank you!
> Chris
>
>> On Sun, May 17, 2015 at 12:17 PM, Clark Haskins <cl...@kafka.guru> wrote:
>>
>> The reassign_partitions znode is the important one. Please paste the
>> contents of it. That node should only exist while there is a reassignment
>> in progress.
>>
>> You can probably fix this up by forcing a new controller to come online by
>> deleting /controller
>>
>> -Clark
>>
>> Sent from my iPhone
>>
>>> On May 17, 2015, at 10:14 AM, Chris Neal <cw...@gmail.com> wrote:
>>>
>>> Hi Clark,
>>>
>>> Thank you for your reply! I do see that znode under /admin:
>>>
>>> root>
>> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
>>> -server myhost.mydomain.com:2181 ls /admin
>>> Connecting to myhost.mydomain.com:2181
>>> <snip>
>>> WATCHER::
>>>
>>> WatchedEvent state:SyncConnected type:None path:null
>>> [reassign_partitions, delete_topics]
>>>
>>> I'm not sure what this tells me though :)
>>> Again, thanks for your time.
>>> Chris
>>>
>>>> On Sun, May 17, 2015 at 12:20 AM, Clark Haskins <cl...@kafka.guru>
>> wrote:
>>>>
>>>> Does the partition reassignment znode exist under /admin in zookeeper?
>>>>
>>>> -Clark
>>>>
>>>> Sent from my iPhone
>>>>
>>>>> On May 16, 2015, at 7:16 PM, Chris Neal <cw...@gmail.com> wrote:
>>>>>
>>>>> Sorry for bumping my own thread. :S Just wanted to get it in front of
>>>> some
>>>>> eyes again!
>>>>>
>>>>> Thanks for your time and help.
>>>>> Chris
>>>>>
>>>>>> On Thu, May 14, 2015 at 10:24 AM, Chris Neal <cw...@gmail.com>
>> wrote:
>>>>>>
>>>>>> Hi All,
>>>>>>
>>>>>> I am running: kafka_2.10-0.8.1.1, and when I run the
>>>>>> reassign-partitions.sh script, I get this:
>>>>>>
>>>>>> Partitions reassignment failed due to Partition reassignment currently
>>>> in
>>>>>> progress for Map(). Aborting operation
>>>>>> kafka.common.AdminCommandFailedException: Partition reassignment
>>>> currently
>>>>>> in progress for Map(). Aborting operation
>>>>>> at
>> kafka.admin.ReassignPartitionsCommand.reassignPartitions(ReassignPartitionsCommand.scala:204)
>>>>>> at
>> kafka.admin.ReassignPartitionsCommand$.executeAssignment(ReassignPartitionsCommand.scala:124)
>>>>>> at
>> kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:49)
>>>>>> at
>> kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
>>>>>>
>>>>>> I'm running it as:
>>>>>> /opt/kafka/kafka/bin/kafka-reassign-partitions.sh --zookeeper
>> $ZOOKEEPER
>>>>>> --reassignment-json-file ./$1.json --execute
>>>>>>
>>>>>> and my json file looks like:
>>>>>> {"version":1,
>>>>>> "partitions":[
>>>>>> {"topic":"perf_mytopic_raw", "partition":0,"replicas":[0,1]},
>>>>>> {"topic":"perf_mytopic_raw","partition":1,"replicas":[0,1]}]}
>>>>>>
>>>>>> The empty Map() looks suspicious, and I found another post from about
>> a
>>>>>> year ago where it was thought that this might be a bug? I looked
>>>> through
>>>>>> JIRA and didn't see anything that looked like this issue was created
>> or
>>>>>> fixed, so I wanted to ask again about it.
>>>>>>
>>>>>> Any ideas why this might be failing? There is definitely not another
>>>>>> reassignment in progress. :)
>>>>>>
>>>>>> Thanks very much for your time.
>>>>>> Chris
>>
Re: Partitions reassignment failed due to Partition reassignment
currently in progress for Map()
Posted by Chris Neal <cw...@gmail.com>.
Sure thing :)
Hopefully I did this right. Somewhat of a Zookeeper noob.
root>
./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
-server myhost.mydomain.com <http://myhost.mydomain.com:2181/>:2181 ls
/admin/reassign_partitions
Connecting to myhost.mydomain.com <http://myhost.mydomain.com:2181/>:2181
<snip>
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
[]
Hope that is helpful :)
If this is not what you were asking for, please just let me know.
Thank you!
Chris
On Sun, May 17, 2015 at 12:17 PM, Clark Haskins <cl...@kafka.guru> wrote:
> The reassign_partitions znode is the important one. Please paste the
> contents of it. That node should only exist while there is a reassignment
> in progress.
>
> You can probably fix this up by forcing a new controller to come online by
> deleting /controller
>
> -Clark
>
> Sent from my iPhone
>
> > On May 17, 2015, at 10:14 AM, Chris Neal <cw...@gmail.com> wrote:
> >
> > Hi Clark,
> >
> > Thank you for your reply! I do see that znode under /admin:
> >
> > root>
> >
> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
> > -server myhost.mydomain.com:2181 ls /admin
> > Connecting to myhost.mydomain.com:2181
> > <snip>
> > WATCHER::
> >
> > WatchedEvent state:SyncConnected type:None path:null
> > [reassign_partitions, delete_topics]
> >
> > I'm not sure what this tells me though :)
> > Again, thanks for your time.
> > Chris
> >
> >> On Sun, May 17, 2015 at 12:20 AM, Clark Haskins <cl...@kafka.guru>
> wrote:
> >>
> >> Does the partition reassignment znode exist under /admin in zookeeper?
> >>
> >> -Clark
> >>
> >> Sent from my iPhone
> >>
> >>> On May 16, 2015, at 7:16 PM, Chris Neal <cw...@gmail.com> wrote:
> >>>
> >>> Sorry for bumping my own thread. :S Just wanted to get it in front of
> >> some
> >>> eyes again!
> >>>
> >>> Thanks for your time and help.
> >>> Chris
> >>>
> >>>> On Thu, May 14, 2015 at 10:24 AM, Chris Neal <cw...@gmail.com>
> wrote:
> >>>>
> >>>> Hi All,
> >>>>
> >>>> I am running: kafka_2.10-0.8.1.1, and when I run the
> >>>> reassign-partitions.sh script, I get this:
> >>>>
> >>>> Partitions reassignment failed due to Partition reassignment currently
> >> in
> >>>> progress for Map(). Aborting operation
> >>>> kafka.common.AdminCommandFailedException: Partition reassignment
> >> currently
> >>>> in progress for Map(). Aborting operation
> >>>> at
> >>
> kafka.admin.ReassignPartitionsCommand.reassignPartitions(ReassignPartitionsCommand.scala:204)
> >>>> at
> >>
> kafka.admin.ReassignPartitionsCommand$.executeAssignment(ReassignPartitionsCommand.scala:124)
> >>>> at
> >>
> kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:49)
> >>>> at
> >>
> kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
> >>>>
> >>>> I'm running it as:
> >>>> /opt/kafka/kafka/bin/kafka-reassign-partitions.sh --zookeeper
> $ZOOKEEPER
> >>>> --reassignment-json-file ./$1.json --execute
> >>>>
> >>>> and my json file looks like:
> >>>> {"version":1,
> >>>> "partitions":[
> >>>> {"topic":"perf_mytopic_raw", "partition":0,"replicas":[0,1]},
> >>>> {"topic":"perf_mytopic_raw","partition":1,"replicas":[0,1]}]}
> >>>>
> >>>> The empty Map() looks suspicious, and I found another post from about
> a
> >>>> year ago where it was thought that this might be a bug? I looked
> >> through
> >>>> JIRA and didn't see anything that looked like this issue was created
> or
> >>>> fixed, so I wanted to ask again about it.
> >>>>
> >>>> Any ideas why this might be failing? There is definitely not another
> >>>> reassignment in progress. :)
> >>>>
> >>>> Thanks very much for your time.
> >>>> Chris
> >>
>
Re: Partitions reassignment failed due to Partition reassignment currently in progress for Map()
Posted by Clark Haskins <cl...@kafka.guru>.
The reassign_partitions znode is the important one. Please paste the contents of it. That node should only exist while there is a reassignment in progress.
You can probably fix this up by forcing a new controller to come online by deleting /controller
-Clark
Sent from my iPhone
> On May 17, 2015, at 10:14 AM, Chris Neal <cw...@gmail.com> wrote:
>
> Hi Clark,
>
> Thank you for your reply! I do see that znode under /admin:
>
> root>
> ./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
> -server myhost.mydomain.com:2181 ls /admin
> Connecting to myhost.mydomain.com:2181
> <snip>
> WATCHER::
>
> WatchedEvent state:SyncConnected type:None path:null
> [reassign_partitions, delete_topics]
>
> I'm not sure what this tells me though :)
> Again, thanks for your time.
> Chris
>
>> On Sun, May 17, 2015 at 12:20 AM, Clark Haskins <cl...@kafka.guru> wrote:
>>
>> Does the partition reassignment znode exist under /admin in zookeeper?
>>
>> -Clark
>>
>> Sent from my iPhone
>>
>>> On May 16, 2015, at 7:16 PM, Chris Neal <cw...@gmail.com> wrote:
>>>
>>> Sorry for bumping my own thread. :S Just wanted to get it in front of
>> some
>>> eyes again!
>>>
>>> Thanks for your time and help.
>>> Chris
>>>
>>>> On Thu, May 14, 2015 at 10:24 AM, Chris Neal <cw...@gmail.com> wrote:
>>>>
>>>> Hi All,
>>>>
>>>> I am running: kafka_2.10-0.8.1.1, and when I run the
>>>> reassign-partitions.sh script, I get this:
>>>>
>>>> Partitions reassignment failed due to Partition reassignment currently
>> in
>>>> progress for Map(). Aborting operation
>>>> kafka.common.AdminCommandFailedException: Partition reassignment
>> currently
>>>> in progress for Map(). Aborting operation
>>>> at
>> kafka.admin.ReassignPartitionsCommand.reassignPartitions(ReassignPartitionsCommand.scala:204)
>>>> at
>> kafka.admin.ReassignPartitionsCommand$.executeAssignment(ReassignPartitionsCommand.scala:124)
>>>> at
>> kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:49)
>>>> at
>> kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
>>>>
>>>> I'm running it as:
>>>> /opt/kafka/kafka/bin/kafka-reassign-partitions.sh --zookeeper $ZOOKEEPER
>>>> --reassignment-json-file ./$1.json --execute
>>>>
>>>> and my json file looks like:
>>>> {"version":1,
>>>> "partitions":[
>>>> {"topic":"perf_mytopic_raw", "partition":0,"replicas":[0,1]},
>>>> {"topic":"perf_mytopic_raw","partition":1,"replicas":[0,1]}]}
>>>>
>>>> The empty Map() looks suspicious, and I found another post from about a
>>>> year ago where it was thought that this might be a bug? I looked
>> through
>>>> JIRA and didn't see anything that looked like this issue was created or
>>>> fixed, so I wanted to ask again about it.
>>>>
>>>> Any ideas why this might be failing? There is definitely not another
>>>> reassignment in progress. :)
>>>>
>>>> Thanks very much for your time.
>>>> Chris
>>
Re: Partitions reassignment failed due to Partition reassignment
currently in progress for Map()
Posted by Chris Neal <cw...@gmail.com>.
Hi Clark,
Thank you for your reply! I do see that znode under /admin:
root>
./opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/zookeeper/bin/zkCli.sh
-server myhost.mydomain.com:2181 ls /admin
Connecting to myhost.mydomain.com:2181
<snip>
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
[reassign_partitions, delete_topics]
I'm not sure what this tells me though :)
Again, thanks for your time.
Chris
On Sun, May 17, 2015 at 12:20 AM, Clark Haskins <cl...@kafka.guru> wrote:
> Does the partition reassignment znode exist under /admin in zookeeper?
>
> -Clark
>
> Sent from my iPhone
>
> > On May 16, 2015, at 7:16 PM, Chris Neal <cw...@gmail.com> wrote:
> >
> > Sorry for bumping my own thread. :S Just wanted to get it in front of
> some
> > eyes again!
> >
> > Thanks for your time and help.
> > Chris
> >
> >> On Thu, May 14, 2015 at 10:24 AM, Chris Neal <cw...@gmail.com> wrote:
> >>
> >> Hi All,
> >>
> >> I am running: kafka_2.10-0.8.1.1, and when I run the
> >> reassign-partitions.sh script, I get this:
> >>
> >> Partitions reassignment failed due to Partition reassignment currently
> in
> >> progress for Map(). Aborting operation
> >> kafka.common.AdminCommandFailedException: Partition reassignment
> currently
> >> in progress for Map(). Aborting operation
> >> at
> >>
> kafka.admin.ReassignPartitionsCommand.reassignPartitions(ReassignPartitionsCommand.scala:204)
> >> at
> >>
> kafka.admin.ReassignPartitionsCommand$.executeAssignment(ReassignPartitionsCommand.scala:124)
> >> at
> >>
> kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:49)
> >> at
> >>
> kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
> >>
> >> I'm running it as:
> >> /opt/kafka/kafka/bin/kafka-reassign-partitions.sh --zookeeper $ZOOKEEPER
> >> --reassignment-json-file ./$1.json --execute
> >>
> >> and my json file looks like:
> >> {"version":1,
> >> "partitions":[
> >> {"topic":"perf_mytopic_raw", "partition":0,"replicas":[0,1]},
> >> {"topic":"perf_mytopic_raw","partition":1,"replicas":[0,1]}]}
> >>
> >> The empty Map() looks suspicious, and I found another post from about a
> >> year ago where it was thought that this might be a bug? I looked
> through
> >> JIRA and didn't see anything that looked like this issue was created or
> >> fixed, so I wanted to ask again about it.
> >>
> >> Any ideas why this might be failing? There is definitely not another
> >> reassignment in progress. :)
> >>
> >> Thanks very much for your time.
> >> Chris
> >>
>
Re: Partitions reassignment failed due to Partition reassignment currently in progress for Map()
Posted by Clark Haskins <cl...@kafka.guru>.
Does the partition reassignment znode exist under /admin in zookeeper?
-Clark
Sent from my iPhone
> On May 16, 2015, at 7:16 PM, Chris Neal <cw...@gmail.com> wrote:
>
> Sorry for bumping my own thread. :S Just wanted to get it in front of some
> eyes again!
>
> Thanks for your time and help.
> Chris
>
>> On Thu, May 14, 2015 at 10:24 AM, Chris Neal <cw...@gmail.com> wrote:
>>
>> Hi All,
>>
>> I am running: kafka_2.10-0.8.1.1, and when I run the
>> reassign-partitions.sh script, I get this:
>>
>> Partitions reassignment failed due to Partition reassignment currently in
>> progress for Map(). Aborting operation
>> kafka.common.AdminCommandFailedException: Partition reassignment currently
>> in progress for Map(). Aborting operation
>> at
>> kafka.admin.ReassignPartitionsCommand.reassignPartitions(ReassignPartitionsCommand.scala:204)
>> at
>> kafka.admin.ReassignPartitionsCommand$.executeAssignment(ReassignPartitionsCommand.scala:124)
>> at
>> kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:49)
>> at
>> kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
>>
>> I'm running it as:
>> /opt/kafka/kafka/bin/kafka-reassign-partitions.sh --zookeeper $ZOOKEEPER
>> --reassignment-json-file ./$1.json --execute
>>
>> and my json file looks like:
>> {"version":1,
>> "partitions":[
>> {"topic":"perf_mytopic_raw", "partition":0,"replicas":[0,1]},
>> {"topic":"perf_mytopic_raw","partition":1,"replicas":[0,1]}]}
>>
>> The empty Map() looks suspicious, and I found another post from about a
>> year ago where it was thought that this might be a bug? I looked through
>> JIRA and didn't see anything that looked like this issue was created or
>> fixed, so I wanted to ask again about it.
>>
>> Any ideas why this might be failing? There is definitely not another
>> reassignment in progress. :)
>>
>> Thanks very much for your time.
>> Chris
>>
Re: Partitions reassignment failed due to Partition reassignment
currently in progress for Map()
Posted by Chris Neal <cw...@gmail.com>.
Sorry for bumping my own thread. :S Just wanted to get it in front of some
eyes again!
Thanks for your time and help.
Chris
On Thu, May 14, 2015 at 10:24 AM, Chris Neal <cw...@gmail.com> wrote:
> Hi All,
>
> I am running: kafka_2.10-0.8.1.1, and when I run the
> reassign-partitions.sh script, I get this:
>
> Partitions reassignment failed due to Partition reassignment currently in
> progress for Map(). Aborting operation
> kafka.common.AdminCommandFailedException: Partition reassignment currently
> in progress for Map(). Aborting operation
> at
> kafka.admin.ReassignPartitionsCommand.reassignPartitions(ReassignPartitionsCommand.scala:204)
> at
> kafka.admin.ReassignPartitionsCommand$.executeAssignment(ReassignPartitionsCommand.scala:124)
> at
> kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:49)
> at
> kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
>
> I'm running it as:
> /opt/kafka/kafka/bin/kafka-reassign-partitions.sh --zookeeper $ZOOKEEPER
> --reassignment-json-file ./$1.json --execute
>
> and my json file looks like:
> {"version":1,
> "partitions":[
> {"topic":"perf_mytopic_raw", "partition":0,"replicas":[0,1]},
> {"topic":"perf_mytopic_raw","partition":1,"replicas":[0,1]}]}
>
> The empty Map() looks suspicious, and I found another post from about a
> year ago where it was thought that this might be a bug? I looked through
> JIRA and didn't see anything that looked like this issue was created or
> fixed, so I wanted to ask again about it.
>
> Any ideas why this might be failing? There is definitely not another
> reassignment in progress. :)
>
> Thanks very much for your time.
> Chris
>