You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by nragon <nu...@wedotechnologies.com> on 2017/07/20 15:51:39 UTC
Detached execution API
It would be nice to let users deploy detached jobs through api.
For instance>
*StreamExecutionEnviroment*
public JobExecutionResult execute() throws Exception {
return execute(DEFAULT_JOB_NAME, false);
}
Which keep backward compatibility
public abstract JobExecutionResult execute(String jobName, boolean
detached) throws Exception;
*RemoteStreamEnvironment*
@Override
public JobExecutionResult execute(String jobName, boolean detached) throws
ProgramInvocationException {
StreamGraph streamGraph = getStreamGraph();
streamGraph.setJobName(jobName);
transformations.clear();
return executeRemotely(streamGraph, jarFiles, detached);
}
---------------------
protected JobExecutionResult executeRemotely(StreamGraph streamGraph,
List<URL> jarFiles, boolean detached) throws ProgramInvocationException {
.....
ClusterClient client;
try {
client = new StandaloneClusterClient(configuration);
client.setDetached(detached);
client.setPrintStatusDuringExecution(getConfig().isSysoutLoggingEnabled());
}
....
}
Just an idea. I'm using the previous approach and it works fine.
Thanks
--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Detached-execution-API-tp14366.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
Re: Detached execution API
Posted by nragon <nu...@wedotechnologies.com>.
Yes, something like that.
Although, this would be an MVP for this purpose which has minimal impacts on
next releases. Moreover, I think clients are too specific to be included as
API because each user will have their own way to implement a stop, start, or
whatever (of course a base client is welcome :)). For instance, we have
implemented one, using REST API + async(detached) job.
Also, one thing I forgot in the previous example is that abstract execute
should return a JobSubmissionResult instead of JobExecutionResult.
But of course, just my point of view.
--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Detached-execution-API-tp14366p14369.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
Re: Detached execution API
Posted by Aljoscha Krettek <al...@apache.org>.
Hi,
I think you might find those two issues interesting:
- https://issues.apache.org/jira/browse/FLINK-2313: Change Streaming Driver Execution Model
- https://issues.apache.org/jira/browse/FLINK-4272: Create a JobClient for job control and monitoring
Best,
Aljoscha
> On 20. Jul 2017, at 17:51, nragon <nu...@wedotechnologies.com> wrote:
>
> It would be nice to let users deploy detached jobs through api.
> For instance>
>
> *StreamExecutionEnviroment*
>
> public JobExecutionResult execute() throws Exception {
> return execute(DEFAULT_JOB_NAME, false);
> }
> Which keep backward compatibility
>
> public abstract JobExecutionResult execute(String jobName, boolean
> detached) throws Exception;
>
> *RemoteStreamEnvironment*
>
> @Override
> public JobExecutionResult execute(String jobName, boolean detached) throws
> ProgramInvocationException {
> StreamGraph streamGraph = getStreamGraph();
> streamGraph.setJobName(jobName);
> transformations.clear();
> return executeRemotely(streamGraph, jarFiles, detached);
> }
>
> ---------------------
>
> protected JobExecutionResult executeRemotely(StreamGraph streamGraph,
> List<URL> jarFiles, boolean detached) throws ProgramInvocationException {
> .....
> ClusterClient client;
> try {
> client = new StandaloneClusterClient(configuration);
> client.setDetached(detached);
>
> client.setPrintStatusDuringExecution(getConfig().isSysoutLoggingEnabled());
> }
> ....
> }
>
> Just an idea. I'm using the previous approach and it works fine.
>
> Thanks
>
>
>
> --
> View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Detached-execution-API-tp14366.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.