You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Heath Ivie <hi...@AutoAnything.com> on 2016/03/31 02:26:27 UTC
RE: Rest Proxy Question
I really need some help on this.
I am able to publish new messages to the topics using the rest proxy.
The issue is that when I query the rest proxy for that topic, even though there is data present, I get "{}" (empty results).
I will get this empty results for some non-deterministic period of time and then out of the blue it will return the results.
I am at a total loss at this point.
If someone has any ideas, I sure would welcome them.
Regards,
Heath
-----Original Message-----
From: Heath Ivie [mailto:hivie@AutoAnything.com]
Sent: Tuesday, February 16, 2016 10:44 AM
To: users@kafka.apache.org
Subject: RE: Rest Proxy Question
Hi Martin,
I can see the items in the queue, that is not the problem.
The issue is when the consumer (using the REST proxy) is trying to consume.
-----Original Message-----
From: Martin Gainty [mailto:mgainty@hotmail.com]
Sent: Tuesday, February 16, 2016 10:32 AM
To: users@kafka.apache.org
Subject: RE: Rest Proxy Question
pathping on the contacted proxy during slack periods | tee slack.lst pathping on the contacted proxy during busy periods | tee busy.lst Please Post Results here
What i am suggesting is once a packet is transmitted outside your firewall the packet is beyond reach of locally hosted software such as Kafka If that is the case you may want to look at Pulling in network engineers to configure arp tables to choose the fastest route I can help you out in case you dont have those resources availableMartin ______________________________________________
_____ _ _____ _ _____ ___ _ _____ _ _ _
|_ _| |_ ___ | _ |___ ___ ___| |_ ___ | __|___| _| |_ _ _ _ ___ ___ ___ | __|___ _ _ ___ _| |___| |_|_|___ ___
| | | | -_| | | . | .'| _| | -_| |__ | . | _| _| | | | .'| _| -_| | __| . | | | | . | .'| _| | . | |
|_| |_|_|___| |__|__| _|__,|___|_|_|___| |_____|___|_| |_| |_____|__,|_| |___| |__| |___|___|_|_|___|__,|_| |_|___|_|_|
|_|
> From: hivie@AutoAnything.com
> To: users@kafka.apache.org
> Subject: Rest Proxy Question
> Date: Tue, 16 Feb 2016 17:28:34 +0000
>
> I am having a hard time with some bug in my code related to the REST proxy.
>
> The issue that I am seeing is when the GET /consumers/HIVIEPurchaseOrderCreatedEvent/instances/b0671fa1-d000-4f9b-b795-aff3003e500a/topics/PurchaseOrderCreatedEvent/.
>
> Sometimes this will return immediately with results and sometimes it takes a very long time with many requests in between when results are received.
>
> I have a job that runs in the background checking for new messages every 5 seconds, so I would expect within a minute or two the messages would be visible.
>
> Have any of you seen this behavior before?
>
> Heath Ivie
> Solutions Architect
>
>
> Warning: This e-mail may contain information proprietary to AutoAnything Inc. and is intended only for the use of the intended recipient(s). If the reader of this message is not the intended recipient(s), you have received this message in error and any review, dissemination, distribution or copying of this message is strictly prohibited. If you have received this message in error, please notify the sender immediately and delete all copies.
RE: Rest Proxy Question
Posted by Heath Ivie <hi...@AutoAnything.com>.
Hi Martin,
I am not using the browser.
I am making the call from code, much like curl.
The proxy is returning a http code 200, but only with an empty json array.
I am not having a delay when calling the proxy, I am having a delay when the proxy is returning results.
Even though there is data in the queue that is ready to be consumed, the proxy returns without results for some non-deterministic period of time.
-----Original Message-----
From: Martin Gainty [mailto:mgainty@hotmail.com]
Sent: Thursday, March 31, 2016 4:59 AM
To: users@kafka.apache.org
Subject: RE: Rest Proxy Question
The issue is when the consumer (using the REST proxy) is trying to consume.MG>can we assume you have disabled all cache between server and client
MG>in fact its probably best now to use a 3rd party tool like curl to pull the data to make sure browser isnt causing the delay:
https://curl.haxx.se/download.html
MG>if you see the same delay
MG>then this is a network issue
MG>fortunately with HTTP 1.1 Server you can implement NIO connections
MG>when server does publish can we see what kafka publisher is publishing?MG>it is also important to understand format of published results html or text or json or xml?
MG>if for no other reason than to trim down the results (return json
MG>instead of xml)
> From: hivie@AutoAnything.com
> To: users@kafka.apache.org
> Subject: RE: Rest Proxy Question
> Date: Thu, 31 Mar 2016 00:26:27 +0000
>
> I really need some help on this.
>
> I am able to publish new messages to the topics using the rest proxy.
>
> The issue is that when I query the rest proxy for that topic, even though there is data present, I get "{}" (empty results).
>
> I will get this empty results for some non-deterministic period of time and then out of the blue it will return the results.
>
> I am at a total loss at this point.
>
> If someone has any ideas, I sure would welcome them.
>
> Regards,
> Heath
>
> -----Original Message-----
> From: Heath Ivie [mailto:hivie@AutoAnything.com]
> Sent: Tuesday, February 16, 2016 10:44 AM
> To: users@kafka.apache.org
> Subject: RE: Rest Proxy Question
>
> Hi Martin,
>
> I can see the items in the queue, that is not the problem.
>
> The issue is when the consumer (using the REST proxy) is trying to consume.
>
> -----Original Message-----
> From: Martin Gainty [mailto:mgainty@hotmail.com]
> Sent: Tuesday, February 16, 2016 10:32 AM
> To: users@kafka.apache.org
> Subject: RE: Rest Proxy Question
>
> pathping on the contacted proxy during slack periods | tee slack.lst
> pathping on the contacted proxy during busy periods | tee busy.lst
> Please Post Results here
>
> What i am suggesting is once a packet is transmitted outside your firewall the packet is beyond reach of locally hosted software such as Kafka If that is the case you may want to look at Pulling in network engineers to configure arp tables to choose the fastest route I can help you out in case you dont have those resources availableMartin ______________________________________________
> _____ _ _____ _ _____ ___ _ _____ _ _ _
> |_ _| |_ ___ | _ |___ ___ ___| |_ ___ | __|___| _| |_ _ _ _ ___ ___ ___ | __|___ _ _ ___ _| |___| |_|_|___ ___
> | | | | -_| | | . | .'| _| | -_| |__ | . | _| _| | | | .'| _| -_| | __| . | | | | . | .'| _| | . | |
> |_| |_|_|___| |__|__| _|__,|___|_|_|___| |_____|___|_| |_| |_____|__,|_| |___| |__| |___|___|_|_|___|__,|_| |_|___|_|_|
> |_|
>
>
> > From: hivie@AutoAnything.com
> > To: users@kafka.apache.org
> > Subject: Rest Proxy Question
> > Date: Tue, 16 Feb 2016 17:28:34 +0000
> >
> > I am having a hard time with some bug in my code related to the REST proxy.
> >
> > The issue that I am seeing is when the GET /consumers/HIVIEPurchaseOrderCreatedEvent/instances/b0671fa1-d000-4f9b-b795-aff3003e500a/topics/PurchaseOrderCreatedEvent/.
> >
> > Sometimes this will return immediately with results and sometimes it takes a very long time with many requests in between when results are received.
> >
> > I have a job that runs in the background checking for new messages every 5 seconds, so I would expect within a minute or two the messages would be visible.
> >
> > Have any of you seen this behavior before?
> >
> > Heath Ivie
> > Solutions Architect
> >
> >
> > Warning: This e-mail may contain information proprietary to AutoAnything Inc. and is intended only for the use of the intended recipient(s). If the reader of this message is not the intended recipient(s), you have received this message in error and any review, dissemination, distribution or copying of this message is strictly prohibited. If you have received this message in error, please notify the sender immediately and delete all copies.
>
RE: Rest Proxy Question
Posted by Martin Gainty <mg...@hotmail.com>.
The issue is when the consumer (using the REST proxy) is trying to consume.MG>can we assume you have disabled all cache between server and client
MG>in fact its probably best now to use a 3rd party tool like curl to pull the data to make sure browser isnt causing the delay:
https://curl.haxx.se/download.html
MG>if you see the same delay
MG>then this is a network issue
MG>fortunately with HTTP 1.1 Server you can implement NIO connections
MG>when server does publish can we see what kafka publisher is publishing?MG>it is also important to understand format of published results html or text or json or xml?
MG>if for no other reason than to trim down the results (return json instead of xml)
> From: hivie@AutoAnything.com
> To: users@kafka.apache.org
> Subject: RE: Rest Proxy Question
> Date: Thu, 31 Mar 2016 00:26:27 +0000
>
> I really need some help on this.
>
> I am able to publish new messages to the topics using the rest proxy.
>
> The issue is that when I query the rest proxy for that topic, even though there is data present, I get "{}" (empty results).
>
> I will get this empty results for some non-deterministic period of time and then out of the blue it will return the results.
>
> I am at a total loss at this point.
>
> If someone has any ideas, I sure would welcome them.
>
> Regards,
> Heath
>
> -----Original Message-----
> From: Heath Ivie [mailto:hivie@AutoAnything.com]
> Sent: Tuesday, February 16, 2016 10:44 AM
> To: users@kafka.apache.org
> Subject: RE: Rest Proxy Question
>
> Hi Martin,
>
> I can see the items in the queue, that is not the problem.
>
> The issue is when the consumer (using the REST proxy) is trying to consume.
>
> -----Original Message-----
> From: Martin Gainty [mailto:mgainty@hotmail.com]
> Sent: Tuesday, February 16, 2016 10:32 AM
> To: users@kafka.apache.org
> Subject: RE: Rest Proxy Question
>
> pathping on the contacted proxy during slack periods | tee slack.lst pathping on the contacted proxy during busy periods | tee busy.lst Please Post Results here
>
> What i am suggesting is once a packet is transmitted outside your firewall the packet is beyond reach of locally hosted software such as Kafka If that is the case you may want to look at Pulling in network engineers to configure arp tables to choose the fastest route I can help you out in case you dont have those resources availableMartin ______________________________________________
> _____ _ _____ _ _____ ___ _ _____ _ _ _
> |_ _| |_ ___ | _ |___ ___ ___| |_ ___ | __|___| _| |_ _ _ _ ___ ___ ___ | __|___ _ _ ___ _| |___| |_|_|___ ___
> | | | | -_| | | . | .'| _| | -_| |__ | . | _| _| | | | .'| _| -_| | __| . | | | | . | .'| _| | . | |
> |_| |_|_|___| |__|__| _|__,|___|_|_|___| |_____|___|_| |_| |_____|__,|_| |___| |__| |___|___|_|_|___|__,|_| |_|___|_|_|
> |_|
>
>
> > From: hivie@AutoAnything.com
> > To: users@kafka.apache.org
> > Subject: Rest Proxy Question
> > Date: Tue, 16 Feb 2016 17:28:34 +0000
> >
> > I am having a hard time with some bug in my code related to the REST proxy.
> >
> > The issue that I am seeing is when the GET /consumers/HIVIEPurchaseOrderCreatedEvent/instances/b0671fa1-d000-4f9b-b795-aff3003e500a/topics/PurchaseOrderCreatedEvent/.
> >
> > Sometimes this will return immediately with results and sometimes it takes a very long time with many requests in between when results are received.
> >
> > I have a job that runs in the background checking for new messages every 5 seconds, so I would expect within a minute or two the messages would be visible.
> >
> > Have any of you seen this behavior before?
> >
> > Heath Ivie
> > Solutions Architect
> >
> >
> > Warning: This e-mail may contain information proprietary to AutoAnything Inc. and is intended only for the use of the intended recipient(s). If the reader of this message is not the intended recipient(s), you have received this message in error and any review, dissemination, distribution or copying of this message is strictly prohibited. If you have received this message in error, please notify the sender immediately and delete all copies.
>