You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Bryan Baugher <bj...@gmail.com> on 2016/08/03 20:21:20 UTC
Unable to write, leader not available
Hi everyone,
I was trying out kerberos on Kafka 0.10.0.0 by creating a single node
cluster. I managed to get everything setup and past all the authentication
errors but whenever I try to use the console producer I get 'Error while
fetching metadata ... LEADER_NOT_AVAILABLE'. In this case I've created the
topic ahead of time (1 replica, 1 partition) and I can see that broker 0 is
in the ISR and is the leader. I have also opened an ACL to the topic for my
user to produce and was previously seeing authentication errors prior. I
don't see any errors or helpful logs on the broker side even after turning
on debug logging. Turning on debug logging on the client the only thing
that stands out is that it lists the broker as 'node -1' instead of 0. It
does mention the correct hostname/port and that it was able to successfully
connect. Any ideas?
Bryan
Re: Unable to write, leader not available
Posted by Bryan Baugher <bj...@gmail.com>.
Found I still hit this issue without VPN. I had to make the cluster's user
a super user or at least give it appropriate privileges
On Thu, Aug 4, 2016 at 11:39 AM Bryan Baugher <bj...@gmail.com> wrote:
> Figured this out. This had to do with me being on a VPN and running
> everything locally
>
> On Thu, Aug 4, 2016 at 11:26 AM Bryan Baugher <bj...@gmail.com> wrote:
>
>> I managed to replicate this issue using the default provided config for
>> kafka/zookeeper in the binary artifact and using the console producer to
>> write a message
>>
>> On Thu, Aug 4, 2016 at 10:23 AM Bryan Baugher <bj...@gmail.com> wrote:
>>
>>> Using the --producer option in kafka-acls.sh it looks to have allowed
>>> create operations on the cluster. Turning on trace logging for
>>> authorization shows repeated mentions of my user and that its allowed to
>>> create on the cluster and describe the topic.
>>>
>>> Looks like I might not be the only one with this issue[1] so I'm
>>> wondering if its not kerberos related
>>>
>>> [1] -
>>> http://mail-archives.apache.org/mod_mbox/kafka-users/201608.mbox/%3CBLU184-W1930FDD3A39241FFDA0E6AB3040%40phx.gbl%3E
>>>
>>> On Wed, Aug 3, 2016 at 11:09 PM Manikumar Reddy <
>>> manikumar.reddy@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> Can you enable Authorization debug logs and check for logs related to
>>>> denied operations..
>>>> we should also enable operations on Cluster resource.
>>>>
>>>>
>>>> Thanks,
>>>> Manikumar
>>>>
>>>> On Thu, Aug 4, 2016 at 1:51 AM, Bryan Baugher <bj...@gmail.com> wrote:
>>>>
>>>> > Hi everyone,
>>>> >
>>>> > I was trying out kerberos on Kafka 0.10.0.0 by creating a single node
>>>> > cluster. I managed to get everything setup and past all the
>>>> authentication
>>>> > errors but whenever I try to use the console producer I get 'Error
>>>> while
>>>> > fetching metadata ... LEADER_NOT_AVAILABLE'. In this case I've
>>>> created the
>>>> > topic ahead of time (1 replica, 1 partition) and I can see that
>>>> broker 0 is
>>>> > in the ISR and is the leader. I have also opened an ACL to the topic
>>>> for my
>>>> > user to produce and was previously seeing authentication errors
>>>> prior. I
>>>> > don't see any errors or helpful logs on the broker side even after
>>>> turning
>>>> > on debug logging. Turning on debug logging on the client the only
>>>> thing
>>>> > that stands out is that it lists the broker as 'node -1' instead of
>>>> 0. It
>>>> > does mention the correct hostname/port and that it was able to
>>>> successfully
>>>> > connect. Any ideas?
>>>> >
>>>> > Bryan
>>>> >
>>>>
>>>
Re: Unable to write, leader not available
Posted by Bryan Baugher <bj...@gmail.com>.
Figured this out. This had to do with me being on a VPN and running
everything locally
On Thu, Aug 4, 2016 at 11:26 AM Bryan Baugher <bj...@gmail.com> wrote:
> I managed to replicate this issue using the default provided config for
> kafka/zookeeper in the binary artifact and using the console producer to
> write a message
>
> On Thu, Aug 4, 2016 at 10:23 AM Bryan Baugher <bj...@gmail.com> wrote:
>
>> Using the --producer option in kafka-acls.sh it looks to have allowed
>> create operations on the cluster. Turning on trace logging for
>> authorization shows repeated mentions of my user and that its allowed to
>> create on the cluster and describe the topic.
>>
>> Looks like I might not be the only one with this issue[1] so I'm
>> wondering if its not kerberos related
>>
>> [1] -
>> http://mail-archives.apache.org/mod_mbox/kafka-users/201608.mbox/%3CBLU184-W1930FDD3A39241FFDA0E6AB3040%40phx.gbl%3E
>>
>> On Wed, Aug 3, 2016 at 11:09 PM Manikumar Reddy <
>> manikumar.reddy@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> Can you enable Authorization debug logs and check for logs related to
>>> denied operations..
>>> we should also enable operations on Cluster resource.
>>>
>>>
>>> Thanks,
>>> Manikumar
>>>
>>> On Thu, Aug 4, 2016 at 1:51 AM, Bryan Baugher <bj...@gmail.com> wrote:
>>>
>>> > Hi everyone,
>>> >
>>> > I was trying out kerberos on Kafka 0.10.0.0 by creating a single node
>>> > cluster. I managed to get everything setup and past all the
>>> authentication
>>> > errors but whenever I try to use the console producer I get 'Error
>>> while
>>> > fetching metadata ... LEADER_NOT_AVAILABLE'. In this case I've created
>>> the
>>> > topic ahead of time (1 replica, 1 partition) and I can see that broker
>>> 0 is
>>> > in the ISR and is the leader. I have also opened an ACL to the topic
>>> for my
>>> > user to produce and was previously seeing authentication errors prior.
>>> I
>>> > don't see any errors or helpful logs on the broker side even after
>>> turning
>>> > on debug logging. Turning on debug logging on the client the only thing
>>> > that stands out is that it lists the broker as 'node -1' instead of 0.
>>> It
>>> > does mention the correct hostname/port and that it was able to
>>> successfully
>>> > connect. Any ideas?
>>> >
>>> > Bryan
>>> >
>>>
>>
Re: Unable to write, leader not available
Posted by Bryan Baugher <bj...@gmail.com>.
I managed to replicate this issue using the default provided config for
kafka/zookeeper in the binary artifact and using the console producer to
write a message
On Thu, Aug 4, 2016 at 10:23 AM Bryan Baugher <bj...@gmail.com> wrote:
> Using the --producer option in kafka-acls.sh it looks to have allowed
> create operations on the cluster. Turning on trace logging for
> authorization shows repeated mentions of my user and that its allowed to
> create on the cluster and describe the topic.
>
> Looks like I might not be the only one with this issue[1] so I'm wondering
> if its not kerberos related
>
> [1] -
> http://mail-archives.apache.org/mod_mbox/kafka-users/201608.mbox/%3CBLU184-W1930FDD3A39241FFDA0E6AB3040%40phx.gbl%3E
>
> On Wed, Aug 3, 2016 at 11:09 PM Manikumar Reddy <ma...@gmail.com>
> wrote:
>
>> Hi,
>>
>> Can you enable Authorization debug logs and check for logs related to
>> denied operations..
>> we should also enable operations on Cluster resource.
>>
>>
>> Thanks,
>> Manikumar
>>
>> On Thu, Aug 4, 2016 at 1:51 AM, Bryan Baugher <bj...@gmail.com> wrote:
>>
>> > Hi everyone,
>> >
>> > I was trying out kerberos on Kafka 0.10.0.0 by creating a single node
>> > cluster. I managed to get everything setup and past all the
>> authentication
>> > errors but whenever I try to use the console producer I get 'Error while
>> > fetching metadata ... LEADER_NOT_AVAILABLE'. In this case I've created
>> the
>> > topic ahead of time (1 replica, 1 partition) and I can see that broker
>> 0 is
>> > in the ISR and is the leader. I have also opened an ACL to the topic
>> for my
>> > user to produce and was previously seeing authentication errors prior. I
>> > don't see any errors or helpful logs on the broker side even after
>> turning
>> > on debug logging. Turning on debug logging on the client the only thing
>> > that stands out is that it lists the broker as 'node -1' instead of 0.
>> It
>> > does mention the correct hostname/port and that it was able to
>> successfully
>> > connect. Any ideas?
>> >
>> > Bryan
>> >
>>
>
Re: Unable to write, leader not available
Posted by Bryan Baugher <bj...@gmail.com>.
Using the --producer option in kafka-acls.sh it looks to have allowed
create operations on the cluster. Turning on trace logging for
authorization shows repeated mentions of my user and that its allowed to
create on the cluster and describe the topic.
Looks like I might not be the only one with this issue[1] so I'm wondering
if its not kerberos related
[1] -
http://mail-archives.apache.org/mod_mbox/kafka-users/201608.mbox/%3CBLU184-W1930FDD3A39241FFDA0E6AB3040%40phx.gbl%3E
On Wed, Aug 3, 2016 at 11:09 PM Manikumar Reddy <ma...@gmail.com>
wrote:
> Hi,
>
> Can you enable Authorization debug logs and check for logs related to
> denied operations..
> we should also enable operations on Cluster resource.
>
>
> Thanks,
> Manikumar
>
> On Thu, Aug 4, 2016 at 1:51 AM, Bryan Baugher <bj...@gmail.com> wrote:
>
> > Hi everyone,
> >
> > I was trying out kerberos on Kafka 0.10.0.0 by creating a single node
> > cluster. I managed to get everything setup and past all the
> authentication
> > errors but whenever I try to use the console producer I get 'Error while
> > fetching metadata ... LEADER_NOT_AVAILABLE'. In this case I've created
> the
> > topic ahead of time (1 replica, 1 partition) and I can see that broker 0
> is
> > in the ISR and is the leader. I have also opened an ACL to the topic for
> my
> > user to produce and was previously seeing authentication errors prior. I
> > don't see any errors or helpful logs on the broker side even after
> turning
> > on debug logging. Turning on debug logging on the client the only thing
> > that stands out is that it lists the broker as 'node -1' instead of 0. It
> > does mention the correct hostname/port and that it was able to
> successfully
> > connect. Any ideas?
> >
> > Bryan
> >
>
Re: Unable to write, leader not available
Posted by Manikumar Reddy <ma...@gmail.com>.
Hi,
Can you enable Authorization debug logs and check for logs related to
denied operations..
we should also enable operations on Cluster resource.
Thanks,
Manikumar
On Thu, Aug 4, 2016 at 1:51 AM, Bryan Baugher <bj...@gmail.com> wrote:
> Hi everyone,
>
> I was trying out kerberos on Kafka 0.10.0.0 by creating a single node
> cluster. I managed to get everything setup and past all the authentication
> errors but whenever I try to use the console producer I get 'Error while
> fetching metadata ... LEADER_NOT_AVAILABLE'. In this case I've created the
> topic ahead of time (1 replica, 1 partition) and I can see that broker 0 is
> in the ISR and is the leader. I have also opened an ACL to the topic for my
> user to produce and was previously seeing authentication errors prior. I
> don't see any errors or helpful logs on the broker side even after turning
> on debug logging. Turning on debug logging on the client the only thing
> that stands out is that it lists the broker as 'node -1' instead of 0. It
> does mention the correct hostname/port and that it was able to successfully
> connect. Any ideas?
>
> Bryan
>