You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by Mike Percy <mp...@apache.org> on 2012/07/17 00:23:17 UTC
Re: Avro remote exception
(adding flume users list, bcc: avro users list)
Shekhar,
Can you please provide more information about what you are doing when you
see this message and the version of Flume that you are running?
Regards,
Mike
On Mon, Jul 16, 2012 at 3:12 PM, Doug Cutting <cu...@apache.org> wrote:
> From the stack trace you sent this looks like an error thrown by
> Flume, not by Avro.
>
> Doug
>
> On Fri, Jul 13, 2012 at 11:53 AM, shekhar sharma <sh...@gmail.com>
> wrote:
> > Hello,
> > i am using avro source to send the events to Flume Avro source.
> Connection
> > is fine but when i am sending the events it is throwing Avro remote
> > exception.
> > I have an application which uses RPC client factory get
> > connected to the avro source. RpcClient is successfuly connected to
> > the Avro Source but when i am sending the events to the Avro source i
> > am getting AvroRemoteException.
> >
> > This is the following code snippet am using:
> > AvroSource source = new AvroSource();
> > Channel channel = new MemoryChannel();
> > List<Channel> ch = new ArrayList();
> > ch.add(channel);
> > ChannelSelector cs = new ReplicatingChannelSelector();
> > cs.setChannels(ch);
> > Configurables.configure(cs, new Context());
> >
> > ChannelProcessor cp = new ChannelProcessor(cs);
> > Context context = new Context();
> > context.put("bind","localhost"
> > );
> > context.put("port", "12345");
> > source.configure(context);
> > source.setChannelProcessor(cp);
> > source.start();
> > //Server server = new NettyServer(new
> > SpecificResponder(AvroSource.class, source), new
> > InetSocketAddress("localhost",12345));
> > //server.start();
> >
> > AvroSourceProtocol client =
> > SpecificRequestor.getClient(AvroSourceProtocol.class, new
> > NettyTransceiver(new InetSocketAddress(12345)));
> > AvroFlumeEvent avroEvent = new AvroFlumeEvent();
> >
> > avroEvent.setHeaders(new HashMap<CharSequence, CharSequence>());
> > avroEvent.setBody(ByteBuffer.wrap("Hello avro".getBytes()));
> > Status status = client.append(avroEvent);
> >
> >
> > Transaction transaction = channel.getTransaction();
> > transaction.begin();
> >
> > Event event1 = channel.take();
> > transaction.commit();
> > transaction.close();
> >
> > And i am getting the following exception:
> >
> > [New I/O server worker #1-1] WARN org.apache.avro.ipc.Responder - user
> error
> > java.lang.NullPointerException
> > at
> >
> org.apache.flume.channel.ChannelProcessor.processEvent(ChannelProcessor.java:183)
> > at org.apache.flume.source.AvroSource.append(AvroSource.java:179)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > at java.lang.reflect.Method.invoke(Method.java:597)
> > at
> >
> org.apache.avro.ipc.specific.SpecificResponder.respond(SpecificResponder.java:88)
> > at org.apache.avro.ipc.Responder.respond(Responder.java:149)
> > at
> >
> org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.messageReceived(NettyServer.java:140)
> > at
> >
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80)
> > at
> >
> org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream(NettyServer.java:125)
> > at
> >
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> > at
> >
> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:783)
> > at
> > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:302)
> > at
> >
> org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:317)
> > at
> >
> org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:299)
> > at
> >
> org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:216)
> > at
> >
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80)
> > at
> >
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> > at
> >
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
> > at
> > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:274)
> > at
> > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:261)
> > at
> > org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:351)
> > at
> >
> org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:282)
> > at
> > org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:202)
> > at
> >
> org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
> > at
> >
> org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > at java.lang.Thread.run(Thread.java:662)
>
Re: Avro remote exception
Posted by Mike Percy <mp...@apache.org>.
Som, on Flume NG you must run a source, channel, and sink all in the same
JVM. An Avro source accepts connections and events from an Avro sink.
Try taking a look @ the user guide:
https://people.apache.org/~mpercy/flume/flume-1.2.0-incubating-SNAPSHOT/docs/FlumeUserGuide.html
Regards,
Mike
On Mon, Jul 16, 2012 at 11:01 PM, shekhar sharma <sh...@gmail.com>wrote:
> Hello,
> I am running on flume-ng (flume-1.2.0.SNAPSHOT) and i am trying to see
> whether the events are being captured by avrosource or not!!!..
>
> When i am running avro source only programmitically it is throwing me this
> error...( I am trying to imitate TestAvroSource.java test)
>
> But if i configure both source and sink in the agent file and then send
> the events, i am not getting any error
>
>
> Well my actuall use case is like this:
>
> machine1---> avro source and memory channel
> machine2--->sink and memory channel
>
> i am trying to achieve this.,
>
> Regards,
> Som Shekhar
>
>
>
>
>
> On Tue, Jul 17, 2012 at 3:53 AM, Mike Percy <mp...@apache.org> wrote:
>
>> (adding flume users list, bcc: avro users list)
>>
>> Shekhar,
>> Can you please provide more information about what you are doing when you
>> see this message and the version of Flume that you are running?
>>
>> Regards,
>> Mike
>>
>>
>>
>> On Mon, Jul 16, 2012 at 3:12 PM, Doug Cutting <cu...@apache.org> wrote:
>>
>>> From the stack trace you sent this looks like an error thrown by
>>> Flume, not by Avro.
>>>
>>> Doug
>>>
>>> On Fri, Jul 13, 2012 at 11:53 AM, shekhar sharma <sh...@gmail.com>
>>> wrote:
>>> > Hello,
>>> > i am using avro source to send the events to Flume Avro source.
>>> Connection
>>> > is fine but when i am sending the events it is throwing Avro remote
>>> > exception.
>>> > I have an application which uses RPC client factory get
>>> > connected to the avro source. RpcClient is successfuly connected to
>>> > the Avro Source but when i am sending the events to the Avro source i
>>> > am getting AvroRemoteException.
>>> >
>>> > This is the following code snippet am using:
>>> > AvroSource source = new AvroSource();
>>> > Channel channel = new MemoryChannel();
>>> > List<Channel> ch = new ArrayList();
>>> > ch.add(channel);
>>> > ChannelSelector cs = new ReplicatingChannelSelector();
>>> > cs.setChannels(ch);
>>> > Configurables.configure(cs, new Context());
>>> >
>>> > ChannelProcessor cp = new ChannelProcessor(cs);
>>> > Context context = new Context();
>>> > context.put("bind","localhost"
>>> > );
>>> > context.put("port", "12345");
>>> > source.configure(context);
>>> > source.setChannelProcessor(cp);
>>> > source.start();
>>> > //Server server = new NettyServer(new
>>> > SpecificResponder(AvroSource.class, source), new
>>> > InetSocketAddress("localhost",12345));
>>> > //server.start();
>>> >
>>> > AvroSourceProtocol client =
>>> > SpecificRequestor.getClient(AvroSourceProtocol.class, new
>>> > NettyTransceiver(new InetSocketAddress(12345)));
>>> > AvroFlumeEvent avroEvent = new AvroFlumeEvent();
>>> >
>>> > avroEvent.setHeaders(new HashMap<CharSequence,
>>> CharSequence>());
>>> > avroEvent.setBody(ByteBuffer.wrap("Hello avro".getBytes()));
>>> > Status status = client.append(avroEvent);
>>> >
>>> >
>>> > Transaction transaction = channel.getTransaction();
>>> > transaction.begin();
>>> >
>>> > Event event1 = channel.take();
>>> > transaction.commit();
>>> > transaction.close();
>>> >
>>> > And i am getting the following exception:
>>> >
>>> > [New I/O server worker #1-1] WARN org.apache.avro.ipc.Responder - user
>>> error
>>> > java.lang.NullPointerException
>>> > at
>>> >
>>> org.apache.flume.channel.ChannelProcessor.processEvent(ChannelProcessor.java:183)
>>> > at
>>> org.apache.flume.source.AvroSource.append(AvroSource.java:179)
>>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> > at
>>> >
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>> > at
>>> >
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> > at java.lang.reflect.Method.invoke(Method.java:597)
>>> > at
>>> >
>>> org.apache.avro.ipc.specific.SpecificResponder.respond(SpecificResponder.java:88)
>>> > at org.apache.avro.ipc.Responder.respond(Responder.java:149)
>>> > at
>>> >
>>> org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.messageReceived(NettyServer.java:140)
>>> > at
>>> >
>>> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80)
>>> > at
>>> >
>>> org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream(NettyServer.java:125)
>>> > at
>>> >
>>> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
>>> > at
>>> >
>>> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:783)
>>> > at
>>> > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:302)
>>> > at
>>> >
>>> org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:317)
>>> > at
>>> >
>>> org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:299)
>>> > at
>>> >
>>> org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:216)
>>> > at
>>> >
>>> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80)
>>> > at
>>> >
>>> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
>>> > at
>>> >
>>> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
>>> > at
>>> > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:274)
>>> > at
>>> > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:261)
>>> > at
>>> > org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:351)
>>> > at
>>> >
>>> org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:282)
>>> > at
>>> > org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:202)
>>> > at
>>> >
>>> org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
>>> > at
>>> >
>>> org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
>>> > at
>>> >
>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>> > at
>>> >
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>> > at java.lang.Thread.run(Thread.java:662)
>>>
>>
>>
>
Re: Avro remote exception
Posted by shekhar sharma <sh...@gmail.com>.
Hello,
I am running on flume-ng (flume-1.2.0.SNAPSHOT) and i am trying to see
whether the events are being captured by avrosource or not!!!..
When i am running avro source only programmitically it is throwing me this
error...( I am trying to imitate TestAvroSource.java test)
But if i configure both source and sink in the agent file and then send the
events, i am not getting any error
Well my actuall use case is like this:
machine1---> avro source and memory channel
machine2--->sink and memory channel
i am trying to achieve this.,
Regards,
Som Shekhar
On Tue, Jul 17, 2012 at 3:53 AM, Mike Percy <mp...@apache.org> wrote:
> (adding flume users list, bcc: avro users list)
>
> Shekhar,
> Can you please provide more information about what you are doing when you
> see this message and the version of Flume that you are running?
>
> Regards,
> Mike
>
>
>
> On Mon, Jul 16, 2012 at 3:12 PM, Doug Cutting <cu...@apache.org> wrote:
>
>> From the stack trace you sent this looks like an error thrown by
>> Flume, not by Avro.
>>
>> Doug
>>
>> On Fri, Jul 13, 2012 at 11:53 AM, shekhar sharma <sh...@gmail.com>
>> wrote:
>> > Hello,
>> > i am using avro source to send the events to Flume Avro source.
>> Connection
>> > is fine but when i am sending the events it is throwing Avro remote
>> > exception.
>> > I have an application which uses RPC client factory get
>> > connected to the avro source. RpcClient is successfuly connected to
>> > the Avro Source but when i am sending the events to the Avro source i
>> > am getting AvroRemoteException.
>> >
>> > This is the following code snippet am using:
>> > AvroSource source = new AvroSource();
>> > Channel channel = new MemoryChannel();
>> > List<Channel> ch = new ArrayList();
>> > ch.add(channel);
>> > ChannelSelector cs = new ReplicatingChannelSelector();
>> > cs.setChannels(ch);
>> > Configurables.configure(cs, new Context());
>> >
>> > ChannelProcessor cp = new ChannelProcessor(cs);
>> > Context context = new Context();
>> > context.put("bind","localhost"
>> > );
>> > context.put("port", "12345");
>> > source.configure(context);
>> > source.setChannelProcessor(cp);
>> > source.start();
>> > //Server server = new NettyServer(new
>> > SpecificResponder(AvroSource.class, source), new
>> > InetSocketAddress("localhost",12345));
>> > //server.start();
>> >
>> > AvroSourceProtocol client =
>> > SpecificRequestor.getClient(AvroSourceProtocol.class, new
>> > NettyTransceiver(new InetSocketAddress(12345)));
>> > AvroFlumeEvent avroEvent = new AvroFlumeEvent();
>> >
>> > avroEvent.setHeaders(new HashMap<CharSequence, CharSequence>());
>> > avroEvent.setBody(ByteBuffer.wrap("Hello avro".getBytes()));
>> > Status status = client.append(avroEvent);
>> >
>> >
>> > Transaction transaction = channel.getTransaction();
>> > transaction.begin();
>> >
>> > Event event1 = channel.take();
>> > transaction.commit();
>> > transaction.close();
>> >
>> > And i am getting the following exception:
>> >
>> > [New I/O server worker #1-1] WARN org.apache.avro.ipc.Responder - user
>> error
>> > java.lang.NullPointerException
>> > at
>> >
>> org.apache.flume.channel.ChannelProcessor.processEvent(ChannelProcessor.java:183)
>> > at
>> org.apache.flume.source.AvroSource.append(AvroSource.java:179)
>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> > at
>> >
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> > at
>> >
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> > at java.lang.reflect.Method.invoke(Method.java:597)
>> > at
>> >
>> org.apache.avro.ipc.specific.SpecificResponder.respond(SpecificResponder.java:88)
>> > at org.apache.avro.ipc.Responder.respond(Responder.java:149)
>> > at
>> >
>> org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.messageReceived(NettyServer.java:140)
>> > at
>> >
>> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80)
>> > at
>> >
>> org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream(NettyServer.java:125)
>> > at
>> >
>> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
>> > at
>> >
>> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:783)
>> > at
>> > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:302)
>> > at
>> >
>> org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:317)
>> > at
>> >
>> org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:299)
>> > at
>> >
>> org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:216)
>> > at
>> >
>> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80)
>> > at
>> >
>> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
>> > at
>> >
>> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
>> > at
>> > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:274)
>> > at
>> > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:261)
>> > at
>> > org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:351)
>> > at
>> >
>> org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:282)
>> > at
>> > org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:202)
>> > at
>> >
>> org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
>> > at
>> >
>> org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
>> > at
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> > at
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> > at java.lang.Thread.run(Thread.java:662)
>>
>
>