You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Avijit Saha <av...@gmail.com> on 2020/07/22 16:57:00 UTC
Re: Docker Taskmanager unable to connect to Flink JpbManager...Connection
RefusedHi,
Hi,
I have built a docker image containing both Flink 1.11 and the job jar as
per instructions at:
https://ci.apache.org/projects/flink/flink-docs-stable/ops/deployment/docker.html
The jobanager starts up fine as follows:
- FLINK_PROPERTIES="jobmanager.rpc.address: 127.0.0.1"
- docker run --env FLINK_PROPERTIES={"${FLINK_PROPERTIES}" -p
0.0.0.0:6123:6123/tcp
flink_with_job_artifacts standalone-job
--job-classname=org.apache.beam.examples.WordCount --runner=FlinkRunner
--inputFile=/opt/flink/conf/flink-conf.yaml --output=/tmp/counts
Now, when launching the taskmanager as follows:
- FLINK_PROPERTIES="jobmanager.rpc.address: 127.0.0.1"
- docker run --env FLINK_PROPERTIES="${FLINK_PROPERTIES}"
flink_with_job_artifacts taskmanager,
the taskmanager fails with the following:
.........
2020-07-22 16:55:25,974 INFO org.apache.flink.runtime.net.ConnectionUtils
[] - Failed to connect from address '/127.0.0.1':
Connection refused (Connection refused)
........
2020-07-22 16:55:32,709 INFO
org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Could
not resolve ResourceManager address akka.tcp://
flink@127.0.0.1:6123/user/rpc/resourcemanager_*, retrying in 10000 ms:
Could not connect to rpc endpoint under address akka.tcp://
flink@127.0.0.1:6123/user/rpc/resourcemanager_*.
Any pointer on How to fix this?
Thanks
Avijit
Re: Docker Taskmanager unable to connect to Flink JpbManager...Connection
RefusedHi,
Posted by Avijit Saha <av...@gmail.com>.
Thanks Yang!
It worked as expected after I made the changes suggested by you!
Avijit
On Wed, Jul 22, 2020 at 11:05 PM Yang Wang <da...@gmail.com> wrote:
> Hi Avijit,
>
> I think you need to create a network via "docker network create
> flink-network".
> And then use "docker run ... --name=jobmanager --network flink-network" to
> set the hostname. Also
> "jobmanager.rpc.address" need to be set the jobmanager. Refer the doc[1]
> for more information.
>
> If you really do not want to create a network and still want to use the
> port forward directly. Then you
> need to set the "jobmanager.rpc.address" to a local ip address, for
> example, 192.168.0.100, not 127.0.0.1.
> Since 127.0.0.1 in docker container means a container local address, not
> host machine local address.
>
>
> [1].
> https://ci.apache.org/projects/flink/flink-docs-stable/ops/deployment/docker.html#start-a-session-cluster
>
> Best,
> Yang
>
> Avijit Saha <av...@gmail.com> 于2020年7月23日周四 上午1:14写道：
>
>> Hi,
>> I have built a docker image containing both Flink 1.11 and the job jar as
>> per instructions at:
>>
>> https://ci.apache.org/projects/flink/flink-docs-stable/ops/deployment/docker.html
>>
>>
>> The jobanager starts up fine as follows:
>>
>> - FLINK_PROPERTIES="jobmanager.rpc.address: 127.0.0.1"
>> - docker run --env FLINK_PROPERTIES={"${FLINK_PROPERTIES}" -p
>> 0.0.0.0:6123:6123/tcp flink_with_job_artifacts standalone-job
>> --job-classname=org.apache.beam.examples.WordCount --runner=FlinkRunner
>> --inputFile=/opt/flink/conf/flink-conf.yaml --output=/tmp/counts
>>
>>
>>
>> Now, when launching the taskmanager as follows:
>> - FLINK_PROPERTIES="jobmanager.rpc.address: 127.0.0.1"
>> - docker run --env FLINK_PROPERTIES="${FLINK_PROPERTIES}"
>> flink_with_job_artifacts taskmanager,
>> the taskmanager fails with the following:
>>
>> .........
>> 2020-07-22 16:55:25,974 INFO
>> org.apache.flink.runtime.net.ConnectionUtils [] - Failed
>> to connect from address '/127.0.0.1': Connection refused (Connection
>> refused)
>> ........
>> 2020-07-22 16:55:32,709 INFO
>> org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Could
>> not resolve ResourceManager address akka.tcp://
>> flink@127.0.0.1:6123/user/rpc/resourcemanager_*, retrying in 10000 ms:
>> Could not connect to rpc endpoint under address akka.tcp://
>> flink@127.0.0.1:6123/user/rpc/resourcemanager_*.
>>
>>
>> Any pointer on How to fix this?
>>
>> Thanks
>> Avijit
>>
>
Re: Docker Taskmanager unable to connect to Flink JpbManager...Connection
RefusedHi,
Posted by Yang Wang <da...@gmail.com>.
Hi Avijit,
I think you need to create a network via "docker network create
flink-network".
And then use "docker run ... --name=jobmanager --network flink-network" to
set the hostname. Also
"jobmanager.rpc.address" need to be set the jobmanager. Refer the doc[1]
for more information.
If you really do not want to create a network and still want to use the
port forward directly. Then you
need to set the "jobmanager.rpc.address" to a local ip address, for
example, 192.168.0.100, not 127.0.0.1.
Since 127.0.0.1 in docker container means a container local address, not
host machine local address.
[1].
https://ci.apache.org/projects/flink/flink-docs-stable/ops/deployment/docker.html#start-a-session-cluster
Best,
Yang
Avijit Saha <av...@gmail.com> 于2020年7月23日周四 上午1:14写道：
> Hi,
> I have built a docker image containing both Flink 1.11 and the job jar as
> per instructions at:
>
> https://ci.apache.org/projects/flink/flink-docs-stable/ops/deployment/docker.html
>
>
> The jobanager starts up fine as follows:
>
> - FLINK_PROPERTIES="jobmanager.rpc.address: 127.0.0.1"
> - docker run --env FLINK_PROPERTIES={"${FLINK_PROPERTIES}" -p 0.0.0.0:6123:6123/tcp
> flink_with_job_artifacts standalone-job
> --job-classname=org.apache.beam.examples.WordCount --runner=FlinkRunner
> --inputFile=/opt/flink/conf/flink-conf.yaml --output=/tmp/counts
>
>
>
> Now, when launching the taskmanager as follows:
> - FLINK_PROPERTIES="jobmanager.rpc.address: 127.0.0.1"
> - docker run --env FLINK_PROPERTIES="${FLINK_PROPERTIES}"
> flink_with_job_artifacts taskmanager,
> the taskmanager fails with the following:
>
> .........
> 2020-07-22 16:55:25,974 INFO org.apache.flink.runtime.net.ConnectionUtils
> [] - Failed to connect from address '/127.0.0.1':
> Connection refused (Connection refused)
> ........
> 2020-07-22 16:55:32,709 INFO
> org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Could
> not resolve ResourceManager address akka.tcp://
> flink@127.0.0.1:6123/user/rpc/resourcemanager_*, retrying in 10000 ms:
> Could not connect to rpc endpoint under address akka.tcp://
> flink@127.0.0.1:6123/user/rpc/resourcemanager_*.
>
>
> Any pointer on How to fix this?
>
> Thanks
> Avijit
>