You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Itai Frenkel <it...@gigaspaces.com> on 2013/04/18 21:29:29 UTC
Is the information at
https://cwiki.apache.org/confluence/display/KAFKA/Operations up to date ?
Hello,
We're considering to use Kafka, and would like to know the Java/ Scala/Kafka version combination in which it is being used in production.
The wiki "Operations" page states that "We are running Kafka 0.7 right now but may move to trunk as we fix bugs."
While I read here that 0.8 was scheduled to be in production in LinkedIn about a month ago.
http://search-hadoop.com/m/4TaT4gjlc12/kafka+production+version&subj=Re+Which+version+to+choose+gt+0+8+or+0+72
Regards,
Itai
Re: Is the information at
https://cwiki.apache.org/confluence/display/KAFKA/Operations up to date ?
Posted by Eric Sites <Er...@threattrack.com>.
Neha,
That makes perfect sense now... :)
The console consumer is working just how I expect it to work now with
that option removed from the command line.
Thanks,
Eric
On 4/19/13 10:31 AM, "Neha Narkhede" <ne...@gmail.com> wrote:
>The reason is the console consumer deletes the previous state information
>from zookeeper when given the from-beginning option. So effectively only
>the 2nd consumer instance is alive. You can run ConsumerShell instead.
>
>Thanks,
>Neha
>
>On Thursday, April 18, 2013, Eric Sites wrote:
>
>> Jun,
>>
>> I have been trying to test 0.8 using two consumers on the same group ID.
>>
>> When I start the second consumer the first stops receiving messages.
>>
>> Can you help?
>>
>> Single Kafka server, single topic, 3 partitions, single Zookeeper.
>>
>> 1 console consumer running in one session.
>>
>> 1 console consumer running in another session.
>>
>> sudo /opt/kafka/bin/kafka-list-topic.sh --zookeeper localhost:2181
>> topic: Files partition: 0 leader: 0 replicas: 0 isr: 0
>> topic: Files partition: 1 leader: 0 replicas: 0 isr: 0
>> topic: Files partition: 2 leader: 0 replicas: 0 isr: 0
>>
>> # Start the producer, and type some test messages
>> sudo /opt/kafka/bin/kafka-console-producer.sh --broker-list
>>localhost:9092
>> --topic Files
>>
>> # Start first consumer
>> sudo /opt/kafka/bin/kafka-console-consumer.sh --zookeeper localhost:2181
>> --topic Files --from-beginning --group 1
>>
>> # Start the second consumer, type some more messages on the producer and
>> only this consumer get the messages
>> sudo /opt/kafka/bin/kafka-console-consumer.sh --zookeeper localhost:2181
>> --topic Files --from-beginning --group 1
>>
>>
>>
>>
>> Thanks,
>> Eric
>>
>> On 4/19/13 12:28 AM, "Jun Rao" <junrao@gmail.com <javascript:;>> wrote:
>>
>> >We have been testing 0.8 at LinkedIn for some time. We plan to release
>>an
>> >0.8 beta version next week.
>> >
>> >Thanks,
>> >
>> >Jun
>> >
>> >
>> >On Thu, Apr 18, 2013 at 12:29 PM, Itai Frenkel
>><itaif@gigaspaces.com<javascript:;>
>> >
>> >wrote:
>> >
>> >> Hello,
>> >>
>> >> We're considering to use Kafka, and would like to know the Java/
>> >> Scala/Kafka version combination in which it is being used in
>>production.
>> >> The wiki "Operations" page states that "We are running Kafka 0.7
>>right
>> >>now
>> >> but may move to trunk as we fix bugs."
>> >>
>> >> While I read here that 0.8 was scheduled to be in production in
>>LinkedIn
>> >> about a month ago.
>> >>
>> >>
>> >>
>>
>>http://search-hadoop.com/m/4TaT4gjlc12/kafka+production+version&subj=Re+W
>> >>hich+version+to+choose+gt+0+8+or+0+72
>> >>
>> >> Regards,
>> >> Itai
>> >>
>>
>>
Re: Is the information at https://cwiki.apache.org/confluence/display/KAFKA/Operations
up to date ?
Posted by Neha Narkhede <ne...@gmail.com>.
The reason is the console consumer deletes the previous state information
from zookeeper when given the from-beginning option. So effectively only
the 2nd consumer instance is alive. You can run ConsumerShell instead.
Thanks,
Neha
On Thursday, April 18, 2013, Eric Sites wrote:
> Jun,
>
> I have been trying to test 0.8 using two consumers on the same group ID.
>
> When I start the second consumer the first stops receiving messages.
>
> Can you help?
>
> Single Kafka server, single topic, 3 partitions, single Zookeeper.
>
> 1 console consumer running in one session.
>
> 1 console consumer running in another session.
>
> sudo /opt/kafka/bin/kafka-list-topic.sh --zookeeper localhost:2181
> topic: Files partition: 0 leader: 0 replicas: 0 isr: 0
> topic: Files partition: 1 leader: 0 replicas: 0 isr: 0
> topic: Files partition: 2 leader: 0 replicas: 0 isr: 0
>
> # Start the producer, and type some test messages
> sudo /opt/kafka/bin/kafka-console-producer.sh --broker-list localhost:9092
> --topic Files
>
> # Start first consumer
> sudo /opt/kafka/bin/kafka-console-consumer.sh --zookeeper localhost:2181
> --topic Files --from-beginning --group 1
>
> # Start the second consumer, type some more messages on the producer and
> only this consumer get the messages
> sudo /opt/kafka/bin/kafka-console-consumer.sh --zookeeper localhost:2181
> --topic Files --from-beginning --group 1
>
>
>
>
> Thanks,
> Eric
>
> On 4/19/13 12:28 AM, "Jun Rao" <junrao@gmail.com <javascript:;>> wrote:
>
> >We have been testing 0.8 at LinkedIn for some time. We plan to release an
> >0.8 beta version next week.
> >
> >Thanks,
> >
> >Jun
> >
> >
> >On Thu, Apr 18, 2013 at 12:29 PM, Itai Frenkel <itaif@gigaspaces.com<javascript:;>
> >
> >wrote:
> >
> >> Hello,
> >>
> >> We're considering to use Kafka, and would like to know the Java/
> >> Scala/Kafka version combination in which it is being used in production.
> >> The wiki "Operations" page states that "We are running Kafka 0.7 right
> >>now
> >> but may move to trunk as we fix bugs."
> >>
> >> While I read here that 0.8 was scheduled to be in production in LinkedIn
> >> about a month ago.
> >>
> >>
> >>
> http://search-hadoop.com/m/4TaT4gjlc12/kafka+production+version&subj=Re+W
> >>hich+version+to+choose+gt+0+8+or+0+72
> >>
> >> Regards,
> >> Itai
> >>
>
>
Re: Is the information at
https://cwiki.apache.org/confluence/display/KAFKA/Operations up to date ?
Posted by Eric Sites <Er...@threattrack.com>.
Jun,
I have been trying to test 0.8 using two consumers on the same group ID.
When I start the second consumer the first stops receiving messages.
Can you help?
Single Kafka server, single topic, 3 partitions, single Zookeeper.
1 console consumer running in one session.
1 console consumer running in another session.
sudo /opt/kafka/bin/kafka-list-topic.sh --zookeeper localhost:2181
topic: Files partition: 0 leader: 0 replicas: 0 isr: 0
topic: Files partition: 1 leader: 0 replicas: 0 isr: 0
topic: Files partition: 2 leader: 0 replicas: 0 isr: 0
# Start the producer, and type some test messages
sudo /opt/kafka/bin/kafka-console-producer.sh --broker-list localhost:9092
--topic Files
# Start first consumer
sudo /opt/kafka/bin/kafka-console-consumer.sh --zookeeper localhost:2181
--topic Files --from-beginning --group 1
# Start the second consumer, type some more messages on the producer and
only this consumer get the messages
sudo /opt/kafka/bin/kafka-console-consumer.sh --zookeeper localhost:2181
--topic Files --from-beginning --group 1
Thanks,
Eric
On 4/19/13 12:28 AM, "Jun Rao" <ju...@gmail.com> wrote:
>We have been testing 0.8 at LinkedIn for some time. We plan to release an
>0.8 beta version next week.
>
>Thanks,
>
>Jun
>
>
>On Thu, Apr 18, 2013 at 12:29 PM, Itai Frenkel <it...@gigaspaces.com>
>wrote:
>
>> Hello,
>>
>> We're considering to use Kafka, and would like to know the Java/
>> Scala/Kafka version combination in which it is being used in production.
>> The wiki "Operations" page states that "We are running Kafka 0.7 right
>>now
>> but may move to trunk as we fix bugs."
>>
>> While I read here that 0.8 was scheduled to be in production in LinkedIn
>> about a month ago.
>>
>>
>>http://search-hadoop.com/m/4TaT4gjlc12/kafka+production+version&subj=Re+W
>>hich+version+to+choose+gt+0+8+or+0+72
>>
>> Regards,
>> Itai
>>
Re: Is the information at https://cwiki.apache.org/confluence/display/KAFKA/Operations
up to date ?
Posted by Jun Rao <ju...@gmail.com>.
We have been testing 0.8 at LinkedIn for some time. We plan to release an
0.8 beta version next week.
Thanks,
Jun
On Thu, Apr 18, 2013 at 12:29 PM, Itai Frenkel <it...@gigaspaces.com> wrote:
> Hello,
>
> We're considering to use Kafka, and would like to know the Java/
> Scala/Kafka version combination in which it is being used in production.
> The wiki "Operations" page states that "We are running Kafka 0.7 right now
> but may move to trunk as we fix bugs."
>
> While I read here that 0.8 was scheduled to be in production in LinkedIn
> about a month ago.
>
> http://search-hadoop.com/m/4TaT4gjlc12/kafka+production+version&subj=Re+Which+version+to+choose+gt+0+8+or+0+72
>
> Regards,
> Itai
>