You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by "Kudumula, Surender" <su...@hpe.com> on 2015/10/20 16:37:52 UTC
Kafka - Rest api query
Dear sir/madam
I have a query. We are working on POC at the moment and we are using kafka to produce and consume messages. I have one component which consumes the request from topic and processes it and creates a file and again produce the java object as byte array to another kafka topic. Now I have two options
1. Is to write a java rest client and send the bytearray java object to rest api service in another component
2. Or I was wondering if its possible to route the message from kafka topic to another component rest APi server.
Please let me know we are using kafka 0.8 and new producer. Any suggestions would be appreciated?
Regards
Surender Kudumula
Big Data Consultant - EMEA
Analytics & Data Management
Surender.kudumula@hpe.com<ma...@hpe.com>
M +44 7795970923
Hewlett-Packard Enterprise
Cain Rd,
Bracknell
RG12 1HN
UK
[http://graphics8.nytimes.com/images/2015/06/03/technology/03bits-hp/03bits-hp-master315.png]
Re: Kafka - Rest api query
Posted by Adam Dubiel <du...@gmail.com>.
I don't think you need to write it from scratch, Hermes project
http://hermes-pubsub.readthedocs.org/en/latest/ does this (and more). You
could probably use only consumers module to change pull to push and push
messages from Kafka to other REST services. It has all the retry and send
rate auto adjusting magic you will probably need at some point.
2015-10-20 23:51 GMT+02:00 Kudumula, Surender <su...@hpe.com>:
> Actually iam planning to write a consumer in a rest client where kafka
> topic is residing and send the object from rest client to another
> webservice which accepts rest api service.
>
> Regards
>
> Surender Kudumula
> Big Data Consultant - EMEA
> Analytics & Data Management
>
> Surender.kudumula@hpe.com
> M +44 7795970923
>
> Hewlett-Packard Enterprise
> Cain Rd,
> Bracknell
> RG12 1HN
> UK
>
>
>
> -----Original Message-----
> From: Selina Tech [mailto:swucareer99@gmail.com]
> Sent: 20 October 2015 21:50
> To: users@kafka.apache.org
> Subject: Re: Kafka - Rest api query
>
> Could you write a consumer at your rest server?
>
> On Tue, Oct 20, 2015 at 1:18 PM, Kudumula, Surender <
> surender.kudumula@hpe.com> wrote:
>
> > Thanks for the reply. Iam looking to know if its possible to route
> > binary objects messages to rest api service from kafka. If so please let
> me know.
> > Otherwise I can consume the binary object using java consumer and then
> > create a rest client and send the binary message via HTTP POST to rest
> > server. Appreciate your suggestions. Thanks
> >
> > Regards
> >
> > Surender Kudumula
> > Big Data Architect - EMEA
> > Analytics & Data Management
> >
> > Surender.kudumula@hp.com
> > M +44 7795970923
> >
> > Hewlett-Packard Company
> > 88 Wood St
> > London EC2V 7QT
> > UK
> >
> >
> >
> > -----Original Message-----
> > From: Sharninder [mailto:sharninder@gmail.com]
> > Sent: 20 October 2015 16:35
> > To: users@kafka.apache.org
> > Subject: Re: Kafka - Rest api query
> >
> > Sounds like an app design decision. What help can this list give you ?
> >
> >
> > > On 20-Oct-2015, at 8:07 PM, Kudumula, Surender <
> > surender.kudumula@hpe.com> wrote:
> > >
> > > Dear sir/madam
> > > I have a query. We are working on POC at the moment and we are using
> > kafka to produce and consume messages. I have one component which
> > consumes the request from topic and processes it and creates a file
> > and again produce the java object as byte array to another kafka
> > topic. Now I have two options
> > > 1. Is to write a java rest client and send the bytearray java
> object
> > to rest api service in another component
> > > 2. Or I was wondering if its possible to route the message from
> > kafka topic to another component rest APi server.
> > >
> > > Please let me know we are using kafka 0.8 and new producer. Any
> > suggestions would be appreciated?
> > >
> > > Regards
> > >
> > > Surender Kudumula
> > > Big Data Consultant - EMEA
> > > Analytics & Data Management
> > >
> > > Surender.kudumula@hpe.com
> > > M +44 7795970923
> > >
> > > Hewlett-Packard Enterprise
> > > Cain Rd,
> > > Bracknell
> > > RG12 1HN
> > > UK
> > >
> > >
> > >
> >
>
RE: Kafka - Rest api query
Posted by "Kudumula, Surender" <su...@hpe.com>.
Actually iam planning to write a consumer in a rest client where kafka topic is residing and send the object from rest client to another webservice which accepts rest api service.
Regards
Surender Kudumula
Big Data Consultant - EMEA
Analytics & Data Management
Surender.kudumula@hpe.com
M +44 7795970923
Hewlett-Packard Enterprise
Cain Rd,
Bracknell
RG12 1HN
UK
-----Original Message-----
From: Selina Tech [mailto:swucareer99@gmail.com]
Sent: 20 October 2015 21:50
To: users@kafka.apache.org
Subject: Re: Kafka - Rest api query
Could you write a consumer at your rest server?
On Tue, Oct 20, 2015 at 1:18 PM, Kudumula, Surender < surender.kudumula@hpe.com> wrote:
> Thanks for the reply. Iam looking to know if its possible to route
> binary objects messages to rest api service from kafka. If so please let me know.
> Otherwise I can consume the binary object using java consumer and then
> create a rest client and send the binary message via HTTP POST to rest
> server. Appreciate your suggestions. Thanks
>
> Regards
>
> Surender Kudumula
> Big Data Architect - EMEA
> Analytics & Data Management
>
> Surender.kudumula@hp.com
> M +44 7795970923
>
> Hewlett-Packard Company
> 88 Wood St
> London EC2V 7QT
> UK
>
>
>
> -----Original Message-----
> From: Sharninder [mailto:sharninder@gmail.com]
> Sent: 20 October 2015 16:35
> To: users@kafka.apache.org
> Subject: Re: Kafka - Rest api query
>
> Sounds like an app design decision. What help can this list give you ?
>
>
> > On 20-Oct-2015, at 8:07 PM, Kudumula, Surender <
> surender.kudumula@hpe.com> wrote:
> >
> > Dear sir/madam
> > I have a query. We are working on POC at the moment and we are using
> kafka to produce and consume messages. I have one component which
> consumes the request from topic and processes it and creates a file
> and again produce the java object as byte array to another kafka
> topic. Now I have two options
> > 1. Is to write a java rest client and send the bytearray java object
> to rest api service in another component
> > 2. Or I was wondering if its possible to route the message from
> kafka topic to another component rest APi server.
> >
> > Please let me know we are using kafka 0.8 and new producer. Any
> suggestions would be appreciated?
> >
> > Regards
> >
> > Surender Kudumula
> > Big Data Consultant - EMEA
> > Analytics & Data Management
> >
> > Surender.kudumula@hpe.com
> > M +44 7795970923
> >
> > Hewlett-Packard Enterprise
> > Cain Rd,
> > Bracknell
> > RG12 1HN
> > UK
> >
> >
> >
>
Re: Kafka - Rest api query
Posted by Selina Tech <sw...@gmail.com>.
Could you write a consumer at your rest server?
On Tue, Oct 20, 2015 at 1:18 PM, Kudumula, Surender <
surender.kudumula@hpe.com> wrote:
> Thanks for the reply. Iam looking to know if its possible to route binary
> objects messages to rest api service from kafka. If so please let me know.
> Otherwise I can consume the binary object using java consumer and then
> create a rest client and send the binary message via HTTP POST to rest
> server. Appreciate your suggestions. Thanks
>
> Regards
>
> Surender Kudumula
> Big Data Architect - EMEA
> Analytics & Data Management
>
> Surender.kudumula@hp.com
> M +44 7795970923
>
> Hewlett-Packard Company
> 88 Wood St
> London EC2V 7QT
> UK
>
>
>
> -----Original Message-----
> From: Sharninder [mailto:sharninder@gmail.com]
> Sent: 20 October 2015 16:35
> To: users@kafka.apache.org
> Subject: Re: Kafka - Rest api query
>
> Sounds like an app design decision. What help can this list give you ?
>
>
> > On 20-Oct-2015, at 8:07 PM, Kudumula, Surender <
> surender.kudumula@hpe.com> wrote:
> >
> > Dear sir/madam
> > I have a query. We are working on POC at the moment and we are using
> kafka to produce and consume messages. I have one component which consumes
> the request from topic and processes it and creates a file and again
> produce the java object as byte array to another kafka topic. Now I have
> two options
> > 1. Is to write a java rest client and send the bytearray java object
> to rest api service in another component
> > 2. Or I was wondering if its possible to route the message from
> kafka topic to another component rest APi server.
> >
> > Please let me know we are using kafka 0.8 and new producer. Any
> suggestions would be appreciated?
> >
> > Regards
> >
> > Surender Kudumula
> > Big Data Consultant - EMEA
> > Analytics & Data Management
> >
> > Surender.kudumula@hpe.com
> > M +44 7795970923
> >
> > Hewlett-Packard Enterprise
> > Cain Rd,
> > Bracknell
> > RG12 1HN
> > UK
> >
> >
> >
>
RE: Kafka - Rest api query
Posted by "Kudumula, Surender" <su...@hpe.com>.
Thanks for the reply. Iam looking to know if its possible to route binary objects messages to rest api service from kafka. If so please let me know. Otherwise I can consume the binary object using java consumer and then create a rest client and send the binary message via HTTP POST to rest server. Appreciate your suggestions. Thanks
Regards
Surender Kudumula
Big Data Architect - EMEA
Analytics & Data Management
Surender.kudumula@hp.com
M +44 7795970923
Hewlett-Packard Company
88 Wood St
London EC2V 7QT
UK
-----Original Message-----
From: Sharninder [mailto:sharninder@gmail.com]
Sent: 20 October 2015 16:35
To: users@kafka.apache.org
Subject: Re: Kafka - Rest api query
Sounds like an app design decision. What help can this list give you ?
> On 20-Oct-2015, at 8:07 PM, Kudumula, Surender <su...@hpe.com> wrote:
>
> Dear sir/madam
> I have a query. We are working on POC at the moment and we are using kafka to produce and consume messages. I have one component which consumes the request from topic and processes it and creates a file and again produce the java object as byte array to another kafka topic. Now I have two options
> 1. Is to write a java rest client and send the bytearray java object to rest api service in another component
> 2. Or I was wondering if its possible to route the message from kafka topic to another component rest APi server.
>
> Please let me know we are using kafka 0.8 and new producer. Any suggestions would be appreciated?
>
> Regards
>
> Surender Kudumula
> Big Data Consultant - EMEA
> Analytics & Data Management
>
> Surender.kudumula@hpe.com
> M +44 7795970923
>
> Hewlett-Packard Enterprise
> Cain Rd,
> Bracknell
> RG12 1HN
> UK
>
>
>
Re: Kafka - Rest api query
Posted by Sharninder <sh...@gmail.com>.
Sounds like an app design decision. What help can this list give you ?
> On 20-Oct-2015, at 8:07 PM, Kudumula, Surender <su...@hpe.com> wrote:
>
> Dear sir/madam
> I have a query. We are working on POC at the moment and we are using kafka to produce and consume messages. I have one component which consumes the request from topic and processes it and creates a file and again produce the java object as byte array to another kafka topic. Now I have two options
> 1. Is to write a java rest client and send the bytearray java object to rest api service in another component
> 2. Or I was wondering if its possible to route the message from kafka topic to another component rest APi server.
>
> Please let me know we are using kafka 0.8 and new producer. Any suggestions would be appreciated?
>
> Regards
>
> Surender Kudumula
> Big Data Consultant - EMEA
> Analytics & Data Management
>
> Surender.kudumula@hpe.com
> M +44 7795970923
>
> Hewlett-Packard Enterprise
> Cain Rd,
> Bracknell
> RG12 1HN
> UK
>
>
>