You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Max Kovgan <ma...@fortscale.com> on 2014/06/15 11:04:56 UTC
username related weirdness with kafka 0.8.1.1 (scala 2.10.1)
hi.
as subj. states, I'm trying to use kafka-0.8.1.1 with samza (0.7).
Initially the work started as a dev. user, e.g. "devuser", and kafka
would run as that user, "and it's been good".
Then, during the deployment attempt, we're trying to run kafka under a
different user, "kafka".
After having that set up, for some reason we get "no leader" problem.
That is the topics are "leaderless".
This is consistent either using the lib from java OR the command line tools.
Our zookeeper comes from cloudera's hadoop (CDH4 and 5 behave the same way).
But alas, when run as devuser the problem does not happen, so I don't
think the problem is zookeeper's version and we did not enable any
zookeeper authentication mechanisms.
I thought the communication with both kafka and zookeeper is done via
a tcp socket, so as long as the connections occur, and corresponding
services can work, there should not be any relation between the
username of either consumer, producer or the service.
Maybe I'm missing something fundamental, but: Is it supposed to behave
like that?
I can provide more "solid" info (like command excerpts, logs, etc. upon request)
Thanks in advance,
Max.
Re: username related weirdness with kafka 0.8.1.1 (scala 2.10.1)
Posted by Joe Stein <jo...@stealth.ly>.
None. What client(s) are you using? Maybe send more logs along for the
error and other info about the setup. If you revert back does everything
work?
On Jun 15, 2014 10:33 AM, "Max Kovgan" <ma...@fortscale.com> wrote:
> Thanks for the response, Joe.
>
> in the non-working setup, kafka user had all the permissions for its own
> directories and log files.
> I ran normal tests via the command line, as user "kafka" and all worked.
>
> I think my question is:
> is there any non-AF_INET socket communication between kafka clients and
> servers ?
> something like shared memory, or (god forbid) same files access !?
>
>
>
>
>
>
> On Sun, Jun 15, 2014 at 5:17 PM, Joe Stein <jo...@stealth.ly> wrote:
>
>> Off the top of my head sounds like directory permission/ownership issue
>> for data.
>> On Jun 15, 2014 5:05 AM, "Max Kovgan" <ma...@fortscale.com> wrote:
>>
>> > hi.
>> > as subj. states, I'm trying to use kafka-0.8.1.1 with samza (0.7).
>> > Initially the work started as a dev. user, e.g. "devuser", and kafka
>> > would run as that user, "and it's been good".
>> >
>> > Then, during the deployment attempt, we're trying to run kafka under a
>> > different user, "kafka".
>> > After having that set up, for some reason we get "no leader" problem.
>> > That is the topics are "leaderless".
>> >
>> > This is consistent either using the lib from java OR the command line
>> > tools.
>> > Our zookeeper comes from cloudera's hadoop (CDH4 and 5 behave the same
>> > way).
>> >
>> > But alas, when run as devuser the problem does not happen, so I don't
>> > think the problem is zookeeper's version and we did not enable any
>> > zookeeper authentication mechanisms.
>> >
>> > I thought the communication with both kafka and zookeeper is done via
>> > a tcp socket, so as long as the connections occur, and corresponding
>> > services can work, there should not be any relation between the
>> > username of either consumer, producer or the service.
>> >
>> >
>> > Maybe I'm missing something fundamental, but: Is it supposed to behave
>> > like that?
>> >
>> > I can provide more "solid" info (like command excerpts, logs, etc. upon
>> > request)
>> >
>> >
>> >
>> > Thanks in advance,
>> > Max.
>> >
>>
>
>
>
> --
>
>
> [image: cid:image010.png@01CE6B4D.2087AF20]
>
>
> [image: cid:image011.png@01CE6B4D.2087AF20]
>
> *MAXIM KOVGAN*
>
> DevOps Engineer
>
>
>
> *www.fortscale.com* <http://www.fortscale.com/>
>
>
>
> US. 1745 Broadway, 17th floor, New York, NY 10019 | Tel: 1 (212) 519-9889
> ISRAEL. 22A Raoul Wallenberg St., Tel Aviv 6971918 | Tel: 972 (3) 600-6078
>
Re: username related weirdness with kafka 0.8.1.1 (scala 2.10.1)
Posted by Max Kovgan <ma...@fortscale.com>.
Thanks for the response, Joe.
in the non-working setup, kafka user had all the permissions for its own
directories and log files.
I ran normal tests via the command line, as user "kafka" and all worked.
I think my question is:
is there any non-AF_INET socket communication between kafka clients and
servers ?
something like shared memory, or (god forbid) same files access !?
On Sun, Jun 15, 2014 at 5:17 PM, Joe Stein <jo...@stealth.ly> wrote:
> Off the top of my head sounds like directory permission/ownership issue
> for data.
> On Jun 15, 2014 5:05 AM, "Max Kovgan" <ma...@fortscale.com> wrote:
>
> > hi.
> > as subj. states, I'm trying to use kafka-0.8.1.1 with samza (0.7).
> > Initially the work started as a dev. user, e.g. "devuser", and kafka
> > would run as that user, "and it's been good".
> >
> > Then, during the deployment attempt, we're trying to run kafka under a
> > different user, "kafka".
> > After having that set up, for some reason we get "no leader" problem.
> > That is the topics are "leaderless".
> >
> > This is consistent either using the lib from java OR the command line
> > tools.
> > Our zookeeper comes from cloudera's hadoop (CDH4 and 5 behave the same
> > way).
> >
> > But alas, when run as devuser the problem does not happen, so I don't
> > think the problem is zookeeper's version and we did not enable any
> > zookeeper authentication mechanisms.
> >
> > I thought the communication with both kafka and zookeeper is done via
> > a tcp socket, so as long as the connections occur, and corresponding
> > services can work, there should not be any relation between the
> > username of either consumer, producer or the service.
> >
> >
> > Maybe I'm missing something fundamental, but: Is it supposed to behave
> > like that?
> >
> > I can provide more "solid" info (like command excerpts, logs, etc. upon
> > request)
> >
> >
> >
> > Thanks in advance,
> > Max.
> >
>
--
[image: cid:image010.png@01CE6B4D.2087AF20]
[image: cid:image011.png@01CE6B4D.2087AF20]
*MAXIM KOVGAN*
DevOps Engineer
*www.fortscale.com* <http://www.fortscale.com/>
US. 1745 Broadway, 17th floor, New York, NY 10019 | Tel: 1 (212) 519-9889
ISRAEL. 22A Raoul Wallenberg St., Tel Aviv 6971918 | Tel: 972 (3) 600-6078
Re: username related weirdness with kafka 0.8.1.1 (scala 2.10.1)
Posted by Joe Stein <jo...@stealth.ly>.
Off the top of my head sounds like directory permission/ownership issue
for data.
On Jun 15, 2014 5:05 AM, "Max Kovgan" <ma...@fortscale.com> wrote:
> hi.
> as subj. states, I'm trying to use kafka-0.8.1.1 with samza (0.7).
> Initially the work started as a dev. user, e.g. "devuser", and kafka
> would run as that user, "and it's been good".
>
> Then, during the deployment attempt, we're trying to run kafka under a
> different user, "kafka".
> After having that set up, for some reason we get "no leader" problem.
> That is the topics are "leaderless".
>
> This is consistent either using the lib from java OR the command line
> tools.
> Our zookeeper comes from cloudera's hadoop (CDH4 and 5 behave the same
> way).
>
> But alas, when run as devuser the problem does not happen, so I don't
> think the problem is zookeeper's version and we did not enable any
> zookeeper authentication mechanisms.
>
> I thought the communication with both kafka and zookeeper is done via
> a tcp socket, so as long as the connections occur, and corresponding
> services can work, there should not be any relation between the
> username of either consumer, producer or the service.
>
>
> Maybe I'm missing something fundamental, but: Is it supposed to behave
> like that?
>
> I can provide more "solid" info (like command excerpts, logs, etc. upon
> request)
>
>
>
> Thanks in advance,
> Max.
>