You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@avro.apache.org by Stefan Krawczyk <st...@nextdoor.com> on 2013/05/22 05:18:10 UTC
Avro RPC: Python to Java isn't working for me...
Hi,
I am trying to use Avro RPC and have a python client talk to a java server,
using the avro-rpc-quickstart <https://github.com/phunt/avro-rpc-quickstart> on
github as a base (I made sure the avro version being pulled in was 1.7.4).
However when I get my python client to talk to the java server I see this
error:
2013-05-20 19:38:32,512 (pool-5-thread-2) [WARN -
org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.exceptionCaught(NettyServer.java:201)]
Unexpected exception from downstream.
org.apache.avro.AvroRuntimeException: Excessively large list allocation
request detected: 539959368 items! Connection closed.
at
org.apache.avro.ipc.NettyTransportCodec$NettyFrameDecoder.decodePackHeader(NettyTransportCodec.java:167)
at
org.apache.avro.ipc.NettyTransportCodec$NettyFrameDecoder.decode(NettyTransportCodec.java:139)
at
org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:286)
at
org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:208)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:94)
at
org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:364)
at
org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:238)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:38)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
>From digging around on the web I understand this is a NettyTransceiver
issue, i.e. the python client isn't using it because it uses the
HTTPTransceiver.
I was wondering, what are my options for moving forward, other than getting
the java server to use the HTTPTransceiver?
Apologies if I have overlooked something that points out what I can do.
Cheers,
Stefan
Re: Avro RPC: Python to Java isn't working for me...
Posted by Stefan Krawczyk <st...@nextdoor.com>.
nope - nobody responded. I guess I'll ask the dev mailing list that
question and see whether they respond.
On Tue, May 28, 2013 at 5:07 PM, Atin Sood <so...@outlook.com> wrote:
> Stefan
>
> Were you ever able to make any progress on this ?
>
> Just curious :)
>
> --
> Atin Sood
> Sent with Sparrow <http://www.sparrowmailapp.com/?sig>
>
> On Wednesday, May 22, 2013 at 7:36 PM, Stefan Krawczyk wrote:
>
> Yes agreed, that would be a good approach to take, however before I go try
> to figure out how to write a NettyTransceiver in python, I'd like to get
> some community input on so that:
> - I understand why there wasn't one built in the first place, and thus
> answer is it actually possible to build that in python
> - I'm not duplicating effort if there is somebody already working on it
>
> So any NettyServer & NettyTransceiver experts in the house?
>
> :)
>
> Cheers,
>
> Stefan
>
>
>
> On Wed, May 22, 2013 at 4:22 PM, Atin Sood <so...@outlook.com> wrote:
>
> Sure Stefan
>
> I am a newbie to both python and avro and have limited experience in
> networking.
>
> But if I am not wrong the problem is at python client end as I am guessing
> you must be using s.th like
>
> client code - attach to the server and send a message
>
>
>
> client = ipc.HTTPTransceiver(server_addr[0], server_addr[1])
>
>
> requestor = ipc.Requestor(PROTOCOL, client)
>
>
>
>
>
>
> So even though you can go ahead and switch to NettyServer in java you will be limited to use httpserver because your python client uses http client
>
>
>
> https://github.com/phunt/avro-rpc-quickstart/blob/master/src/main/java/example/Main.java
>
>
>
> I guess the best way to get around this will be to look at source code of avro python client ipc.py file
>
>
>
> and add a new implementation besides the one that comes out of the box.
>
>
>
> class HTTPTransceiver(object):
> """
> A simple HTTP-based transceiver implementation.
> Useful for clients but not for servers
> """
>
>
> That said, again I am new to the whole thing so I might be totally wrong :)
>
>
> --
> Atin Sood
> Sent with Sparrow <http://www.sparrowmailapp.com/?sig>
>
> On Wednesday, May 22, 2013 at 11:55 AM, Stefan Krawczyk wrote:
>
> Hi Atin,
>
> Thanks for the response. Yes I understand I could use HTTPServer on the
> java side and things would work. However I'm after a solution where I can
> still have the java side use the NettyServer.
>
> Cheers,
>
> Stefan
>
>
> On Wed, May 22, 2013 at 4:11 AM, Atin Sood <so...@outlook.com> wrote:
>
> You can try looking into something that I wrote as an example
>
>
> https://github.com/atinsood/HESDataAnalyticsFinalProject/tree/master/javaXPython
>
> https://github.com/atinsood/HESDataAnalyticsFinalProject#javaxpython
>
> --
> Atin Sood
> Sent with Sparrow <http://www.sparrowmailapp.com/?sig>
>
> On Tuesday, May 21, 2013 at 11:18 PM, Stefan Krawczyk wrote:
>
> Hi,
>
> I am trying to use Avro RPC and have a python client talk to a java
> server, using the avro-rpc-quickstart<https://github.com/phunt/avro-rpc-quickstart> on
> github as a base (I made sure the avro version being pulled in was 1.7.4).
> However when I get my python client to talk to the java server I see this
> error:
>
> 2013-05-20 19:38:32,512 (pool-5-thread-2) [WARN -
> org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.exceptionCaught(NettyServer.java:201)]
> Unexpected exception from downstream.
> org.apache.avro.AvroRuntimeException: Excessively large list allocation
> request detected: 539959368 items! Connection closed.
> at
> org.apache.avro.ipc.NettyTransportCodec$NettyFrameDecoder.decodePackHeader(NettyTransportCodec.java:167)
> at
> org.apache.avro.ipc.NettyTransportCodec$NettyFrameDecoder.decode(NettyTransportCodec.java:139)
> at
> org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:286)
> at
> org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:208)
> at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
> at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
> at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:94)
> at
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:364)
> at
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:238)
> at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:38)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:722)
>
> From digging around on the web I understand this is a NettyTransceiver
> issue, i.e. the python client isn't using it because it uses the
> HTTPTransceiver.
>
> I was wondering, what are my options for moving forward, other than
> getting the java server to use the HTTPTransceiver?
>
> Apologies if I have overlooked something that points out what I can do.
>
> Cheers,
>
> Stefan
>
>
>
>
>
>
>
Re: Avro RPC: Python to Java isn't working for me...
Posted by Atin Sood <so...@outlook.com>.
Stefan
Were you ever able to make any progress on this ?
Just curious :)
--
Atin Sood
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
On Wednesday, May 22, 2013 at 7:36 PM, Stefan Krawczyk wrote:
> Yes agreed, that would be a good approach to take, however before I go try to figure out how to write a NettyTransceiver in python, I'd like to get some community input on so that:
> - I understand why there wasn't one built in the first place, and thus answer is it actually possible to build that in python
> - I'm not duplicating effort if there is somebody already working on it
>
> So any NettyServer & NettyTransceiver experts in the house?
>
> :)
>
> Cheers,
>
> Stefan
>
>
>
> On Wed, May 22, 2013 at 4:22 PM, Atin Sood <soodatin@outlook.com (mailto:soodatin@outlook.com)> wrote:
> > Sure Stefan
> >
> > I am a newbie to both python and avro and have limited experience in networking.
> >
> > But if I am not wrong the problem is at python client end as I am guessing you must be using s.th (http://s.th) like
> >
> > client code - attach to the server and send a message
> > client = ipc.HTTPTransceiver(server_addr[0], server_addr[1])
> > requestor = ipc.Requestor(PROTOCOL, client)
> >
> >
> > So even though you can go ahead and switch to NettyServer in java you will be limited to use httpserver because your python client uses http client
> >
> > https://github.com/phunt/avro-rpc-quickstart/blob/master/src/main/java/example/Main.java
> >
> > I guess the best way to get around this will be to look at source code of avro python client ipc.py file
> >
> > and add a new implementation besides the one that comes out of the box.
> >
> > class HTTPTransceiver(object): """ A simple HTTP-based transceiver implementation. Useful for clients but not for servers """
> >
> >
> > That said, again I am new to the whole thing so I might be totally wrong :)
> >
> > --
> > Atin Sood
> > Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
> >
> >
> > On Wednesday, May 22, 2013 at 11:55 AM, Stefan Krawczyk wrote:
> >
> > > Hi Atin,
> > >
> > > Thanks for the response. Yes I understand I could use HTTPServer on the java side and things would work. However I'm after a solution where I can still have the java side use the NettyServer.
> > >
> > > Cheers,
> > >
> > > Stefan
> > >
> > >
> > > On Wed, May 22, 2013 at 4:11 AM, Atin Sood <soodatin@outlook.com (mailto:soodatin@outlook.com)> wrote:
> > > > You can try looking into something that I wrote as an example
> > > >
> > > > https://github.com/atinsood/HESDataAnalyticsFinalProject/tree/master/javaXPython
> > > >
> > > > https://github.com/atinsood/HESDataAnalyticsFinalProject#javaxpython
> > > >
> > > > --
> > > > Atin Sood
> > > > Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
> > > >
> > > >
> > > > On Tuesday, May 21, 2013 at 11:18 PM, Stefan Krawczyk wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I am trying to use Avro RPC and have a python client talk to a java server, using the avro-rpc-quickstart (https://github.com/phunt/avro-rpc-quickstart) on github as a base (I made sure the avro version being pulled in was 1.7.4). However when I get my python client to talk to the java server I see this error:
> > > > >
> > > > > 2013-05-20 19:38:32,512 (pool-5-thread-2) [WARN - org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.exceptionCaught(NettyServer.java:201)] Unexpected exception from downstream.
> > > > > org.apache.avro.AvroRuntimeException: Excessively large list allocation request detected: 539959368 items! Connection closed.
> > > > > at org.apache.avro.ipc.NettyTransportCodec$NettyFrameDecoder.decodePackHeader(NettyTransportCodec.java:167)
> > > > > at org.apache.avro.ipc.NettyTransportCodec$NettyFrameDecoder.decode(NettyTransportCodec.java:139)
> > > > > at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:286)
> > > > > at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:208)
> > > > > at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
> > > > > at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
> > > > > at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:94)
> > > > > at org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:364)
> > > > > at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:238)
> > > > > at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:38)
> > > > > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> > > > > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> > > > > at java.lang.Thread.run(Thread.java:722)
> > > > >
> > > > > From digging around on the web I understand this is a NettyTransceiver issue, i.e. the python client isn't using it because it uses the HTTPTransceiver.
> > > > >
> > > > > I was wondering, what are my options for moving forward, other than getting the java server to use the HTTPTransceiver?
> > > > >
> > > > > Apologies if I have overlooked something that points out what I can do.
> > > > >
> > > > > Cheers,
> > > > >
> > > > > Stefan
> > > >
> > >
> >
>
Re: Avro RPC: Python to Java isn't working for me...
Posted by Stefan Krawczyk <st...@nextdoor.com>.
Yes agreed, that would be a good approach to take, however before I go try
to figure out how to write a NettyTransceiver in python, I'd like to get
some community input on so that:
- I understand why there wasn't one built in the first place, and thus
answer is it actually possible to build that in python
- I'm not duplicating effort if there is somebody already working on it
So any NettyServer & NettyTransceiver experts in the house?
:)
Cheers,
Stefan
On Wed, May 22, 2013 at 4:22 PM, Atin Sood <so...@outlook.com> wrote:
> Sure Stefan
>
> I am a newbie to both python and avro and have limited experience in
> networking.
>
> But if I am not wrong the problem is at python client end as I am guessing
> you must be using s.th like
>
> client code - attach to the server and send a message
>
> client = ipc.HTTPTransceiver(server_addr[0], server_addr[1])
> requestor = ipc.Requestor(PROTOCOL, client)
>
>
> So even though you can go ahead and switch to NettyServer in java you will be limited to use httpserver because your python client uses http client
>
> https://github.com/phunt/avro-rpc-quickstart/blob/master/src/main/java/example/Main.java
>
> I guess the best way to get around this will be to look at source code of avro python client ipc.py file
>
> and add a new implementation besides the one that comes out of the box.
>
> class HTTPTransceiver(object):
> """
> A simple HTTP-based transceiver implementation.
> Useful for clients but not for servers
> """
>
>
> That said, again I am new to the whole thing so I might be totally wrong :)
>
>
> --
> Atin Sood
> Sent with Sparrow <http://www.sparrowmailapp.com/?sig>
>
> On Wednesday, May 22, 2013 at 11:55 AM, Stefan Krawczyk wrote:
>
> Hi Atin,
>
> Thanks for the response. Yes I understand I could use HTTPServer on the
> java side and things would work. However I'm after a solution where I can
> still have the java side use the NettyServer.
>
> Cheers,
>
> Stefan
>
>
> On Wed, May 22, 2013 at 4:11 AM, Atin Sood <so...@outlook.com> wrote:
>
> You can try looking into something that I wrote as an example
>
>
> https://github.com/atinsood/HESDataAnalyticsFinalProject/tree/master/javaXPython
>
> https://github.com/atinsood/HESDataAnalyticsFinalProject#javaxpython
>
> --
> Atin Sood
> Sent with Sparrow <http://www.sparrowmailapp.com/?sig>
>
> On Tuesday, May 21, 2013 at 11:18 PM, Stefan Krawczyk wrote:
>
> Hi,
>
> I am trying to use Avro RPC and have a python client talk to a java
> server, using the avro-rpc-quickstart<https://github.com/phunt/avro-rpc-quickstart> on
> github as a base (I made sure the avro version being pulled in was 1.7.4).
> However when I get my python client to talk to the java server I see this
> error:
>
> 2013-05-20 19:38:32,512 (pool-5-thread-2) [WARN -
> org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.exceptionCaught(NettyServer.java:201)]
> Unexpected exception from downstream.
> org.apache.avro.AvroRuntimeException: Excessively large list allocation
> request detected: 539959368 items! Connection closed.
> at
> org.apache.avro.ipc.NettyTransportCodec$NettyFrameDecoder.decodePackHeader(NettyTransportCodec.java:167)
> at
> org.apache.avro.ipc.NettyTransportCodec$NettyFrameDecoder.decode(NettyTransportCodec.java:139)
> at
> org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:286)
> at
> org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:208)
> at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
> at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
> at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:94)
> at
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:364)
> at
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:238)
> at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:38)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:722)
>
> From digging around on the web I understand this is a NettyTransceiver
> issue, i.e. the python client isn't using it because it uses the
> HTTPTransceiver.
>
> I was wondering, what are my options for moving forward, other than
> getting the java server to use the HTTPTransceiver?
>
> Apologies if I have overlooked something that points out what I can do.
>
> Cheers,
>
> Stefan
>
>
>
>
>
Re: Avro RPC: Python to Java isn't working for me...
Posted by Atin Sood <so...@outlook.com>.
Sure Stefan
I am a newbie to both python and avro and have limited experience in networking.
But if I am not wrong the problem is at python client end as I am guessing you must be using s.th like
client code - attach to the server and send a message
client = ipc.HTTPTransceiver(server_addr[0], server_addr[1])
requestor = ipc.Requestor(PROTOCOL, client)
So even though you can go ahead and switch to NettyServer in java you will be limited to use httpserver because your python client uses http client
https://github.com/phunt/avro-rpc-quickstart/blob/master/src/main/java/example/Main.java
I guess the best way to get around this will be to look at source code of avro python client ipc.py file
and add a new implementation besides the one that comes out of the box.
class HTTPTransceiver(object): """ A simple HTTP-based transceiver implementation. Useful for clients but not for servers """
That said, again I am new to the whole thing so I might be totally wrong :)
--
Atin Sood
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
On Wednesday, May 22, 2013 at 11:55 AM, Stefan Krawczyk wrote:
> Hi Atin,
>
> Thanks for the response. Yes I understand I could use HTTPServer on the java side and things would work. However I'm after a solution where I can still have the java side use the NettyServer.
>
> Cheers,
>
> Stefan
>
>
> On Wed, May 22, 2013 at 4:11 AM, Atin Sood <soodatin@outlook.com (mailto:soodatin@outlook.com)> wrote:
> > You can try looking into something that I wrote as an example
> >
> > https://github.com/atinsood/HESDataAnalyticsFinalProject/tree/master/javaXPython
> >
> > https://github.com/atinsood/HESDataAnalyticsFinalProject#javaxpython
> >
> > --
> > Atin Sood
> > Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
> >
> >
> > On Tuesday, May 21, 2013 at 11:18 PM, Stefan Krawczyk wrote:
> >
> > > Hi,
> > >
> > > I am trying to use Avro RPC and have a python client talk to a java server, using the avro-rpc-quickstart (https://github.com/phunt/avro-rpc-quickstart) on github as a base (I made sure the avro version being pulled in was 1.7.4). However when I get my python client to talk to the java server I see this error:
> > >
> > > 2013-05-20 19:38:32,512 (pool-5-thread-2) [WARN - org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.exceptionCaught(NettyServer.java:201)] Unexpected exception from downstream.
> > > org.apache.avro.AvroRuntimeException: Excessively large list allocation request detected: 539959368 items! Connection closed.
> > > at org.apache.avro.ipc.NettyTransportCodec$NettyFrameDecoder.decodePackHeader(NettyTransportCodec.java:167)
> > > at org.apache.avro.ipc.NettyTransportCodec$NettyFrameDecoder.decode(NettyTransportCodec.java:139)
> > > at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:286)
> > > at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:208)
> > > at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
> > > at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
> > > at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:94)
> > > at org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:364)
> > > at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:238)
> > > at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:38)
> > > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> > > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> > > at java.lang.Thread.run(Thread.java:722)
> > >
> > > From digging around on the web I understand this is a NettyTransceiver issue, i.e. the python client isn't using it because it uses the HTTPTransceiver.
> > >
> > > I was wondering, what are my options for moving forward, other than getting the java server to use the HTTPTransceiver?
> > >
> > > Apologies if I have overlooked something that points out what I can do.
> > >
> > > Cheers,
> > >
> > > Stefan
> >
>
Re: Avro RPC: Python to Java isn't working for me...
Posted by Stefan Krawczyk <st...@nextdoor.com>.
Hi Atin,
Thanks for the response. Yes I understand I could use HTTPServer on the
java side and things would work. However I'm after a solution where I can
still have the java side use the NettyServer.
Cheers,
Stefan
On Wed, May 22, 2013 at 4:11 AM, Atin Sood <so...@outlook.com> wrote:
> You can try looking into something that I wrote as an example
>
>
> https://github.com/atinsood/HESDataAnalyticsFinalProject/tree/master/javaXPython
>
> https://github.com/atinsood/HESDataAnalyticsFinalProject#javaxpython
>
> --
> Atin Sood
> Sent with Sparrow <http://www.sparrowmailapp.com/?sig>
>
> On Tuesday, May 21, 2013 at 11:18 PM, Stefan Krawczyk wrote:
>
> Hi,
>
> I am trying to use Avro RPC and have a python client talk to a java
> server, using the avro-rpc-quickstart<https://github.com/phunt/avro-rpc-quickstart> on
> github as a base (I made sure the avro version being pulled in was 1.7.4).
> However when I get my python client to talk to the java server I see this
> error:
>
> 2013-05-20 19:38:32,512 (pool-5-thread-2) [WARN -
> org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.exceptionCaught(NettyServer.java:201)]
> Unexpected exception from downstream.
> org.apache.avro.AvroRuntimeException: Excessively large list allocation
> request detected: 539959368 items! Connection closed.
> at
> org.apache.avro.ipc.NettyTransportCodec$NettyFrameDecoder.decodePackHeader(NettyTransportCodec.java:167)
> at
> org.apache.avro.ipc.NettyTransportCodec$NettyFrameDecoder.decode(NettyTransportCodec.java:139)
> at
> org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:286)
> at
> org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:208)
> at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
> at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
> at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:94)
> at
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:364)
> at
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:238)
> at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:38)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:722)
>
> From digging around on the web I understand this is a NettyTransceiver
> issue, i.e. the python client isn't using it because it uses the
> HTTPTransceiver.
>
> I was wondering, what are my options for moving forward, other than
> getting the java server to use the HTTPTransceiver?
>
> Apologies if I have overlooked something that points out what I can do.
>
> Cheers,
>
> Stefan
>
>
>
Re: Avro RPC: Python to Java isn't working for me...
Posted by Atin Sood <so...@outlook.com>.
You can try looking into something that I wrote as an example
https://github.com/atinsood/HESDataAnalyticsFinalProject/tree/master/javaXPython
https://github.com/atinsood/HESDataAnalyticsFinalProject#javaxpython
--
Atin Sood
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
On Tuesday, May 21, 2013 at 11:18 PM, Stefan Krawczyk wrote:
> Hi,
>
> I am trying to use Avro RPC and have a python client talk to a java server, using the avro-rpc-quickstart (https://github.com/phunt/avro-rpc-quickstart) on github as a base (I made sure the avro version being pulled in was 1.7.4). However when I get my python client to talk to the java server I see this error:
>
> 2013-05-20 19:38:32,512 (pool-5-thread-2) [WARN - org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.exceptionCaught(NettyServer.java:201)] Unexpected exception from downstream.
> org.apache.avro.AvroRuntimeException: Excessively large list allocation request detected: 539959368 items! Connection closed.
> at org.apache.avro.ipc.NettyTransportCodec$NettyFrameDecoder.decodePackHeader(NettyTransportCodec.java:167)
> at org.apache.avro.ipc.NettyTransportCodec$NettyFrameDecoder.decode(NettyTransportCodec.java:139)
> at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:286)
> at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:208)
> at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
> at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
> at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:94)
> at org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:364)
> at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:238)
> at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:38)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:722)
>
> From digging around on the web I understand this is a NettyTransceiver issue, i.e. the python client isn't using it because it uses the HTTPTransceiver.
>
> I was wondering, what are my options for moving forward, other than getting the java server to use the HTTPTransceiver?
>
> Apologies if I have overlooked something that points out what I can do.
>
> Cheers,
>
> Stefan