Flume exception when using SSL

["Needham, Guy" <Gu...@virginmedia.co.uk>]

Hi,

I'm attempting to use Flume to send data using Avro between two servers. The process is running perfectly fine until I started to use SSL encryption. I'm sending data from server A to server B.

Say on server B I make a keystore using:

keytool -genkey -alias serverB -keyalg RSA -keystore keystore.jks -keysize 2048

Then get a certificate using

keytool -export -alias serverB -file serverB.crt -keystore keystore.jks

Then on server A I import this certificate to the truststore:

keytool -import -file serverB.crt -alias serverB -keystore trust.store

On server A I change the conf to include:

agent_foo.sinks.avro-forward-sink.ssl = true
agent_foo.sinks.avro-forward-sink.truststore = trust.store
agent_foo.sinks.avro-forward-sink.truststore-password = serverApassword

On server B I change the conf to include:

a1.sources.sr1.ssl = true
a1.sources.sr1.keystore = keystore.jks
a1.sources.sr1.keystore-password = serverBpassword

When I start the agents, I can see that a connection was opened between server A and server B.

When I try to send data between them, I get this exception on server B:

2014-06-13 12:12:28,301 (pool-7-thread-3) [WARN - org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.exceptionCaught(NettyServer.java:201)] Unexpected exception from downstream.
org.apache.avro.AvroRuntimeException: Excessively large list allocation request detected: 1124139008 items! Connection closed.
        at org.apache.avro.ipc.NettyTransportCodec$NettyFrameDecoder.decodePackHeader(NettyTransportCodec.java:167)
        at org.apache.avro.ipc.NettyTransportCodec$NettyFrameDecoder.decode(NettyTransportCodec.java:139)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:286)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:208)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
        at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:94)
        at org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:364)
        at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:238)
        at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:38)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919)
        at java.lang.Thread.run(Thread.java:738)

Can anyone help me with this exception? I expect that I may have created the certificates incorrectly.

Regards,

Guy Needham | Data Discovery
Virgin Media | Enterprise Data, Design & Management
Bartley Wood Business Park, Hook, Hampshire RG27 9UP
D 01256 75 3362

I welcome VSRE emails. Learn more at http://vsre.info/



--------------------------------------------------------------------
Save Paper - Do you really need to print this e-mail?

Visit www.virginmedia.com for more information, and more fun.

This email and any attachments are or may be confidential and legally privileged
and are sent solely for the attention of the addressee(s). If you have received this
email in error, please delete it from your system: its use, disclosure or copying is
unauthorised. Statements and opinions expressed in this email may not represent
those of Virgin Media. Any representations or commitments in this email are
subject to contract. 

Registered office: Media House, Bartley Wood Business Park, Hook, Hampshire, RG27 9UP
Registered in England and Wales with number 2591237