Can't send data to another service in addSink

[Duy Truong <du...@gmail.com>]

Hi

In my flink program, I want to send data to a server via MQTT protocol,
here is my code

https://gist.github.com/duytruong/d240958d2c4140b554b604cbef05edc7

The problem is it ran well on IntelliJ, but when submitted to cluster
(bin/start-local.sh), it could connect but failed to send data. When I've
tried to move connectToMqttServer() to addSink(), it worked (on cluster),
so I have 2 questions:

1. Why it could connect but failed to send data, I guess the cause is
addSink() is executed in different thread from the connectToMqttServer's
thread but I can't find any document about it.

2. Why it ran well in IntelliJ but failed on cluster (I didn't move
connectToMqttServer() to addSink() in this case).

Thanks,

-- 
*Duy Truong*

Re: Can't send data to another service in addSink

[Chesnay Schepler <ch...@apache.org>]

Please read the Basic API concepts guide in the documentation, in 
particular 
https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/api_concepts.html#lazy-evaluation.

The short answer is that main() is called on the client, while the sink 
is executed on a taskmanager, i.e. in a different JVM. The sink must be 
thus self-contained, i.e. it must call connectToMqttServer().

On 18.09.2017 06:00, Duy Truong wrote:
> Hi
>
> In my flink program, I want to send data to a server via MQTT 
> protocol, here is my code||
>
> https://gist.github.com/duytruong/d240958d2c4140b554b604cbef05edc7 
> <https://gist.github.com/duytruong/d240958d2c4140b554b604cbef05edc7>
>
> The problem is it ran well on IntelliJ, but when submitted to cluster 
> (bin/start-local.sh), it could connect but failed to send data. When 
> I've tried to move connectToMqttServer() to addSink(), it worked (on 
> cluster), so I have 2 questions:
>
> 1. Why it could connect but failed to send data, I guess the cause is 
> addSink() is executed in different thread from the 
> connectToMqttServer's thread but I can't find any document about it.
>
> 2. Why it ran well in IntelliJ but failed on cluster (I didn't move 
> connectToMqttServer() to addSink() in this case).
>
> Thanks,
>
> -- 
> /Duy Truong/