help on producing from local laptop to ec2 via jump

[Robert Towne <rt...@janrain.com>]

Hi, all.  I am trying to produce on my local dev box to an ec2 broker via
an ec2 jump server.  I have tried with the following topology below using
port forwarding w/no success:

*Setup*
local workstation:              local
EC2 Jump Server:            server1
EC2 Broker :                     server2

If I ssh from local to server2 I do so like this:

ssh -o StrictHostKeyChecking=no -o ProxyCommand="ssh -W %h:%p ubunt@server1"
> ubuntu@server2


What I would like to do is instead of simply ssh, I would like to produce
from local to server2.

I have tried this via port forwarding:

ssh -v -Nntf -o ProxyCommand="ssh -W %h:%p ubuntu@server1" -L
> 9092:localhost:9092 ubuntu@server2
> Server2 / broker / server.properties file has:
> listeners=PLAINTEXT://0.0.0.0:9092
> advertised.listeners=PLAINTEXT://server2:9092


My producer on local is writing to localhost:9092

When I try to write,

val message = new ProducerRecord[String, String]("topic", *key*, json)
> val sent = producer.send(message)
> val metadata = sent.get()
>

It fails on the sent.get() line with this erro:

Exception in thread "main" java.util.concurrent.ExecutionException:
> org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for
> topic-0: 30036 ms has passed since batch creation plus linger time
> at
> org.apache.kafka.clients.producer.internals.FutureRecordMetadata.valueOrError(FutureRecordMetadata.java:70)



There are no errors in the broker server.log.
The ssh verbose output looks like this when starting:

debug1: Authentication succeeded (publickey).
> Authenticated to server2 (via proxy).
> debug1: Local connections to LOCALHOST:9092 forwarded to remote address
> localhost:9092
> debug1: Local forwarding listening on ::1 port 9092.
> debug1: channel 0: new [port listener]
> debug1: Local forwarding listening on 127.0.0.1 port 9092.
> debug1: channel 1: new [port listener]
> debug1: Requesting no-more-sessions@openssh.com
> debug1: forking to background
> debug1: Entering interactive session.
> debug1: pledge: proc


and when in code the producer sends the message I see:

> debug1: Connection to port 9092 forwarding to localhost port 9092
> requested.
> debug1: channel 2: new [direct-tcpip]
> debug1: channel 2: free: direct-tcpip: listening port 9092 for localhost
> port 9092, connect from 127.0.0.1 port 64791 to 127.0.0.1 port 9092,
> nchannels 3


Has anyone had success trying this scenario before?  Any suggestions on
where else to look where the error is happening?


Thank you very much!

Robert