You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Otis Gospodnetic <ot...@gmail.com> on 2014/01/31 22:30:51 UTC
C++ Producer => Broker => Java Consumer?
Hi,
If Kafka Producer is using a C++ Kafka lib to produce messages, how can
Kafka Consumers written in Java deserialize them?
Thanks,
Otis
--
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/
Re: C++ Producer => Broker => Java Consumer?
Posted by David Arthur <mu...@gmail.com>.
If you're working with more complex messages than strings, any of the
dearth of cross-language serialization frameworks will work.
On 1/31/14 5:51 PM, Otis Gospodnetic wrote:
> Beautiful then! I thought this cause problems with Java consumer not
> knowing how to deserialize, but sounds like I don't have to worry.
> Excellent, thanks!
>
> Otis
> --
> Performance Monitoring * Log Analytics * Search Analytics
> Solr & Elasticsearch Support * http://sematext.com/
>
>
> On Fri, Jan 31, 2014 at 5:43 PM, Philip O'Toole <ph...@loggly.com> wrote:
>
>> Exactly.
>>
>> Our C++ producers simply stream bytes to 0.72 Kafka, following Kafka's
>> byte-level message spec. Our Java-based Consumers just read bytes and use
>> the standard IO libraries to deserialize the data.
>>
>> Philip
>>
>>
>> On Fri, Jan 31, 2014 at 2:38 PM, Tom Brown <to...@gmail.com> wrote:
>>
>>> The C++ program writes bytes to kafka, and java reads bytes from kafka.
>>>
>>> Is there something special about the way the messages are being
>> serialized
>>> in C++?
>>>
>>> --Tom
>>>
>>>
>>> On Fri, Jan 31, 2014 at 2:36 PM, Philip O'Toole <ph...@loggly.com>
>> wrote:
>>>> Is this a Kafka C++ lib you wrote yourself, or some open-source
>> library?
>>>> What version of Kafka?
>>>>
>>>> Philip
>>>>
>>>>
>>>> On Fri, Jan 31, 2014 at 1:30 PM, Otis Gospodnetic <
>>>> otis.gospodnetic@gmail.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> If Kafka Producer is using a C++ Kafka lib to produce messages, how
>> can
>>>>> Kafka Consumers written in Java deserialize them?
>>>>>
>>>>> Thanks,
>>>>> Otis
>>>>> --
>>>>> Performance Monitoring * Log Analytics * Search Analytics
>>>>> Solr & Elasticsearch Support * http://sematext.com/
>>>>>
Re: C++ Producer => Broker => Java Consumer?
Posted by Otis Gospodnetic <ot...@gmail.com>.
Beautiful then! I thought this cause problems with Java consumer not
knowing how to deserialize, but sounds like I don't have to worry.
Excellent, thanks!
Otis
--
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/
On Fri, Jan 31, 2014 at 5:43 PM, Philip O'Toole <ph...@loggly.com> wrote:
> Exactly.
>
> Our C++ producers simply stream bytes to 0.72 Kafka, following Kafka's
> byte-level message spec. Our Java-based Consumers just read bytes and use
> the standard IO libraries to deserialize the data.
>
> Philip
>
>
> On Fri, Jan 31, 2014 at 2:38 PM, Tom Brown <to...@gmail.com> wrote:
>
> > The C++ program writes bytes to kafka, and java reads bytes from kafka.
> >
> > Is there something special about the way the messages are being
> serialized
> > in C++?
> >
> > --Tom
> >
> >
> > On Fri, Jan 31, 2014 at 2:36 PM, Philip O'Toole <ph...@loggly.com>
> wrote:
> >
> > > Is this a Kafka C++ lib you wrote yourself, or some open-source
> library?
> > > What version of Kafka?
> > >
> > > Philip
> > >
> > >
> > > On Fri, Jan 31, 2014 at 1:30 PM, Otis Gospodnetic <
> > > otis.gospodnetic@gmail.com> wrote:
> > >
> > > > Hi,
> > > >
> > > > If Kafka Producer is using a C++ Kafka lib to produce messages, how
> can
> > > > Kafka Consumers written in Java deserialize them?
> > > >
> > > > Thanks,
> > > > Otis
> > > > --
> > > > Performance Monitoring * Log Analytics * Search Analytics
> > > > Solr & Elasticsearch Support * http://sematext.com/
> > > >
> > >
> >
>
Re: C++ Producer => Broker => Java Consumer?
Posted by Philip O'Toole <ph...@loggly.com>.
Exactly.
Our C++ producers simply stream bytes to 0.72 Kafka, following Kafka's
byte-level message spec. Our Java-based Consumers just read bytes and use
the standard IO libraries to deserialize the data.
Philip
On Fri, Jan 31, 2014 at 2:38 PM, Tom Brown <to...@gmail.com> wrote:
> The C++ program writes bytes to kafka, and java reads bytes from kafka.
>
> Is there something special about the way the messages are being serialized
> in C++?
>
> --Tom
>
>
> On Fri, Jan 31, 2014 at 2:36 PM, Philip O'Toole <ph...@loggly.com> wrote:
>
> > Is this a Kafka C++ lib you wrote yourself, or some open-source library?
> > What version of Kafka?
> >
> > Philip
> >
> >
> > On Fri, Jan 31, 2014 at 1:30 PM, Otis Gospodnetic <
> > otis.gospodnetic@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > If Kafka Producer is using a C++ Kafka lib to produce messages, how can
> > > Kafka Consumers written in Java deserialize them?
> > >
> > > Thanks,
> > > Otis
> > > --
> > > Performance Monitoring * Log Analytics * Search Analytics
> > > Solr & Elasticsearch Support * http://sematext.com/
> > >
> >
>
Re: C++ Producer => Broker => Java Consumer?
Posted by Tom Brown <to...@gmail.com>.
The C++ program writes bytes to kafka, and java reads bytes from kafka.
Is there something special about the way the messages are being serialized
in C++?
--Tom
On Fri, Jan 31, 2014 at 2:36 PM, Philip O'Toole <ph...@loggly.com> wrote:
> Is this a Kafka C++ lib you wrote yourself, or some open-source library?
> What version of Kafka?
>
> Philip
>
>
> On Fri, Jan 31, 2014 at 1:30 PM, Otis Gospodnetic <
> otis.gospodnetic@gmail.com> wrote:
>
> > Hi,
> >
> > If Kafka Producer is using a C++ Kafka lib to produce messages, how can
> > Kafka Consumers written in Java deserialize them?
> >
> > Thanks,
> > Otis
> > --
> > Performance Monitoring * Log Analytics * Search Analytics
> > Solr & Elasticsearch Support * http://sematext.com/
> >
>
Re: C++ Producer => Broker => Java Consumer?
Posted by Philip O'Toole <ph...@loggly.com>.
Is this a Kafka C++ lib you wrote yourself, or some open-source library?
What version of Kafka?
Philip
On Fri, Jan 31, 2014 at 1:30 PM, Otis Gospodnetic <
otis.gospodnetic@gmail.com> wrote:
> Hi,
>
> If Kafka Producer is using a C++ Kafka lib to produce messages, how can
> Kafka Consumers written in Java deserialize them?
>
> Thanks,
> Otis
> --
> Performance Monitoring * Log Analytics * Search Analytics
> Solr & Elasticsearch Support * http://sematext.com/
>