You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Biswajit Das <bi...@gmail.com> on 2017/08/15 01:04:03 UTC
Automate Job submission with container
Hi There,
A few weeks back I have posted here regarding flink docker container
running on mesos. I'm able to run the same successfully; I'm heading
towards full CI/CD deployment with the marathon, most of our deployment is
automated via the marathon, and sometimes even I trigger metrics based auto
scale from the marathon or another scaling service or move service with AWS
spot instance randomly.
Now how do I trigger the job automatically inside the container once JM &
TM is up . I have tested few option and just wanted to validate with the
group here. I have uber app jar already bundled inside the Docker container
.
Option .
1. Schedule a one-time cron like task inside the container with flink
run -d and some delay.
2. Submit via REST, but it complicates the process as I have to find app
DNS from my CI although I have this setup with Bamboo and HAProxy just want
to avoid this complicated process.( given I have many apps running )
I'm sure someone already came across the similar problem, please let me
know if any one has any inputs or suggestion.
~ Biswajit
Re: Automate Job submission with container
Posted by Mar_zieh <m....@gmail.com>.
I want to run my flink program on Mesos cluster via marathon. I created an
application with this Json file in Marathon:
{
"id": "flink",
"cmd": "/home/flink-1.7.0/bin/mesos-appmaster.sh
-Djobmanager.heap.mb=1024 -Djobmanager.rpc.port=6123 -Drest.port=8081
-Dmesos.resourcemanager.tasks.mem=1024 -Dtaskmanager.heap.mb=1024
-Dtaskmanager.numberOfTaskSlots=2 -Dparallelism.default=2
-Dmesos.resourcemanager.tasks.cpus=1",
"cpus": 1.0,
"mem": 1024
}
The task became failed with this error:
I0303 09:41:52.841243 2594 exec.cpp:162] Version: 1.7.0
I0303 09:41:52.851898 2593 exec.cpp:236] Executor registered on agent
d9a98175-b93c-4600-a41b-fe91fae5486a-S0
I0303 09:41:52.854436 2594 executor.cpp:182] Received SUBSCRIBED event
I0303 09:41:52.855284 2594 executor.cpp:186] Subscribed executor on
172.28.10.136
I0303 09:41:52.855479 2594 executor.cpp:182] Received LAUNCH event
I0303 09:41:52.855932 2594 executor.cpp:679] Starting task
ffff.933fdd2f-3d98-11e9-bbc4-0242a78449af
I0303 09:41:52.868172 2594 executor.cpp:499] Running
'/home/mesos-1.7.0/build/src/mesos-containerizer launch
<POSSIBLY-SENSITIVE-DATA>'
I0303 09:41:52.872699 2594 executor.cpp:693] Forked command at 2599
I0303 09:41:54.050284 2596 executor.cpp:994] Command exited with status 1
(pid: 2599)
I0303 09:41:55.052323 2598 process.cpp:926] Stopped the socket accept loop
I configured Zookeeper, Mesos, Marathon and Flink. Moreover, they are all on
docker. I ran a simple program like "echo "hello" >> /home/output.txt"
without any problems.
I really do not know what is going on, I am confused. Would you please any
one tell me what is wrong here?
Any help would be appreciated.
Many thanks.
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Re: Automate Job submission with container
Posted by Mar_zieh <m....@gmail.com>.
I want to run my flink program on Mesos cluster via marathon. I created an
application with this Json file in Marathon:
{
"id": "flink",
"cmd": "/home/flink-1.7.0/bin/mesos-appmaster.sh
-Djobmanager.heap.mb=1024 -Djobmanager.rpc.port=6123 -Drest.port=8081
-Dmesos.resourcemanager.tasks.mem=1024 -Dtaskmanager.heap.mb=1024
-Dtaskmanager.numberOfTaskSlots=2 -Dparallelism.default=2
-Dmesos.resourcemanager.tasks.cpus=1",
"cpus": 1.0,
"mem": 1024
}
The task became failed with this error:
I0303 09:41:52.841243 2594 exec.cpp:162] Version: 1.7.0
I0303 09:41:52.851898 2593 exec.cpp:236] Executor registered on agent
d9a98175-b93c-4600-a41b-fe91fae5486a-S0
I0303 09:41:52.854436 2594 executor.cpp:182] Received SUBSCRIBED event
I0303 09:41:52.855284 2594 executor.cpp:186] Subscribed executor on
172.28.10.136
I0303 09:41:52.855479 2594 executor.cpp:182] Received LAUNCH event
I0303 09:41:52.855932 2594 executor.cpp:679] Starting task
ffff.933fdd2f-3d98-11e9-bbc4-0242a78449af
I0303 09:41:52.868172 2594 executor.cpp:499] Running
'/home/mesos-1.7.0/build/src/mesos-containerizer launch
<POSSIBLY-SENSITIVE-DATA>'
I0303 09:41:52.872699 2594 executor.cpp:693] Forked command at 2599
I0303 09:41:54.050284 2596 executor.cpp:994] Command exited with status 1
(pid: 2599)
I0303 09:41:55.052323 2598 process.cpp:926] Stopped the socket accept loop
I configured Zookeeper, Mesos, Marathon and Flink. Moreover, they are all on
docker. I ran a simple program like "echo "hello" >> /home/output.txt"
without any problems.
I really do not know what is going on, I am confused. Would you please any
one tell me what is wrong here?
Any help would be appreciated.
Many thanks.
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Re: Automate Job submission with container
Posted by Mar_zieh <m....@gmail.com>.
I want to run my flink program on Mesos cluster via marathon. I created an
application with this Json file in Marathon:
{
"id": "flink",
"cmd": "/home/flink-1.7.0/bin/mesos-appmaster.sh
-Djobmanager.heap.mb=1024 -Djobmanager.rpc.port=6123 -Drest.port=8081
-Dmesos.resourcemanager.tasks.mem=1024 -Dtaskmanager.heap.mb=1024
-Dtaskmanager.numberOfTaskSlots=2 -Dparallelism.default=2
-Dmesos.resourcemanager.tasks.cpus=1",
"cpus": 1.0,
"mem": 1024
}
The task became failed with this error:
I0303 09:41:52.841243 2594 exec.cpp:162] Version: 1.7.0
I0303 09:41:52.851898 2593 exec.cpp:236] Executor registered on agent
d9a98175-b93c-4600-a41b-fe91fae5486a-S0
I0303 09:41:52.854436 2594 executor.cpp:182] Received SUBSCRIBED event
I0303 09:41:52.855284 2594 executor.cpp:186] Subscribed executor on
172.28.10.136
I0303 09:41:52.855479 2594 executor.cpp:182] Received LAUNCH event
I0303 09:41:52.855932 2594 executor.cpp:679] Starting task
ffff.933fdd2f-3d98-11e9-bbc4-0242a78449af
I0303 09:41:52.868172 2594 executor.cpp:499] Running
'/home/mesos-1.7.0/build/src/mesos-containerizer launch
<POSSIBLY-SENSITIVE-DATA>'
I0303 09:41:52.872699 2594 executor.cpp:693] Forked command at 2599
I0303 09:41:54.050284 2596 executor.cpp:994] Command exited with status 1
(pid: 2599)
I0303 09:41:55.052323 2598 process.cpp:926] Stopped the socket accept loop
I configured Zookeeper, Mesos, Marathon and Flink. Moreover, they are all on
docker. I ran a simple program like "echo "hello" >> /home/output.txt"
without any problems.
I really do not know what is going on, I am confused. Would you please any
one tell me what is wrong here?
Any help would be appreciated.
Many thanks.
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Re: Automate Job submission with container
Posted by Eron Wright <er...@gmail.com>.
Flink 1.3 relies on a two-step approach of deploying the cluster on Mesos
and then deploying the job. The cron task seems like a good approach;
maybe retry until the job is successfully deployed (using the 'detached'
option assumedly). One complication is that the slots take some time to
come online, and the JM will reject the job submission until sufficient
slots are available.
One of the goals of the FLIP-6 improvement project is to allow for a
one-step deployment of cluster plus job. In the meantime please use a
workaround like you described.
Thanks
On Mon, Aug 14, 2017 at 6:04 PM, Biswajit Das <bi...@gmail.com> wrote:
> Hi There,
>
> A few weeks back I have posted here regarding flink docker container
> running on mesos. I'm able to run the same successfully; I'm heading
> towards full CI/CD deployment with the marathon, most of our deployment is
> automated via the marathon, and sometimes even I trigger metrics based auto
> scale from the marathon or another scaling service or move service with AWS
> spot instance randomly.
> Now how do I trigger the job automatically inside the container once JM &
> TM is up . I have tested few option and just wanted to validate with the
> group here. I have uber app jar already bundled inside the Docker container
> .
>
> Option .
> 1. Schedule a one-time cron like task inside the container with flink
> run -d and some delay.
> 2. Submit via REST, but it complicates the process as I have to find app
> DNS from my CI although I have this setup with Bamboo and HAProxy just want
> to avoid this complicated process.( given I have many apps running )
>
> I'm sure someone already came across the similar problem, please let me
> know if any one has any inputs or suggestion.
>
> ~ Biswajit
>