getting error after upgrade Flink 1.11.1

[dasraj <ra...@microfocus.com>]

Hi,

Currently we are using Flink 1.9.2 in our project and we are submitting Job
from jar.
below is that code we are using currently for jobsubmission:

clusterClient.setDetached(true);
clusterClient.run(packagedProgram,
Integer.valueOf(params.get(Constants.PARAMS_PARALLELISM)));

After upgrading it to Flink 1.11.1, above piece of code is not working.
Also new packagedProgram constructor is expecting "Configuration" as
parameter.

How can we change that to work for our needs?


Thanks,



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: getting error after upgrade Flink 1.11.1

[Kostas Kloudas <kk...@gmail.com>]

Hi Dasraj,

Yes, I would recommend to use Public and, if necessary, PublicEvolving
APIs as they provide better guarantees for future maintenance.
Unfortunately there are no Docs about which APIs are public or
publiceEvolving but you can see the annotations of the classes in the
source code.
I guess you have access to the source code given that previously you
were using the cluster client directly.

You can always of course use Flink's REST API [1] to submit a job,
which provides stability guarantees.

If you want to risk a bit more, you can check the PipelineExecutors in
the codebase to see how job submission is currently done within Flink.
This will give you some ideas I hope about how to proceed.

I hope this helps,
Kostas

[1]https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/rest_api.html

On Fri, Aug 7, 2020 at 8:07 AM dasraj <ra...@microfocus.com> wrote:
>
> Hi Kostas,
>
> I am trying to migrate our code base to use new ClusterClient method for job
> submission.
> As you recommending to use new publicEvolving APIs, any doc or link for
> reference will be helpful.
>
> Thanks,
>
>
>
>
> --
> Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: getting error after upgrade Flink 1.11.1

[dasraj <ra...@microfocus.com>]

Hi Kostas,

I am trying to migrate our code base to use new ClusterClient method for job
submission.
As you recommending to use new publicEvolving APIs, any doc or link for
reference will be helpful.

Thanks,




--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: getting error after upgrade Flink 1.11.1

[Kostas Kloudas <kk...@gmail.com>]

Hi Dasraj,

You are right. On your previous email I did not pay attention that you
migrated from 1.9.
Since 1.9 the ClusterClient has changed significantly as it is not
annotated as @Public API.

I am not sure how easy it is to use the old logic in your settings.
You could try copying the old code but I would recommend migrating to
the new codebase and use public or publicEvolving APIs. This will make
it easier for you to maintain your code in the future, as these APIs
come with stronger backwards compatibility guarantees.

Cheers,
Kostas


On Tue, Aug 4, 2020 at 2:40 PM dasraj <ra...@microfocus.com> wrote:
>
> Hi Kostas,
>
> ClusterClient class is not our custom class it is in
> org.apache.flink.client.program package.
> i can see defination of ClusterClient has changed from 1.9.2 to 1.11.1
>
> https://ci.apache.org/projects/flink/flink-docs-release-1.9/api/java/org/apache/flink/client/program/ClusterClient.html
> <https://ci.apache.org/projects/flink/flink-docs-release-1.9/api/java/org/apache/flink/client/program/ClusterClient.html>
>
> https://ci.apache.org/projects/flink/flink-docs-release-1.11/api/java/org/apache/flink/client/program/ClusterClient.html
> <https://ci.apache.org/projects/flink/flink-docs-release-1.11/api/java/org/apache/flink/client/program/ClusterClient.html>
>
>
> Thanks,
>
>
>
>
>
> --
> Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: getting error after upgrade Flink 1.11.1

[dasraj <ra...@microfocus.com>]

Hi Kostas,

ClusterClient class is not our custom class it is in
org.apache.flink.client.program package.
i can see defination of ClusterClient has changed from 1.9.2 to 1.11.1

https://ci.apache.org/projects/flink/flink-docs-release-1.9/api/java/org/apache/flink/client/program/ClusterClient.html
<https://ci.apache.org/projects/flink/flink-docs-release-1.9/api/java/org/apache/flink/client/program/ClusterClient.html>  

https://ci.apache.org/projects/flink/flink-docs-release-1.11/api/java/org/apache/flink/client/program/ClusterClient.html
<https://ci.apache.org/projects/flink/flink-docs-release-1.11/api/java/org/apache/flink/client/program/ClusterClient.html>  


Thanks,





--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: getting error after upgrade Flink 1.11.1

[Kostas Kloudas <kk...@gmail.com>]

Hi Dasraj,

Could you please specify where is the clusterClient.run() method and
how does it submit a job to a cluster?
Is the clusterClient your custom code?

Any details will help us pin down the problem.
One thing that is worth looking at is the release-notes of 1.11 [1].
There you will find all the user-facing changes that came with the
release.
For example, one note that may be relevant to you is [2].

Cheers,
Kostas

[1] https://ci.apache.org/projects/flink/flink-docs-stable/release-notes/flink-1.11.html
[2] https://ci.apache.org/projects/flink/flink-docs-stable/release-notes/flink-1.11.html#web-submission-behaves-the-same-as-detached-mode

On Tue, Aug 4, 2020 at 9:07 AM dasraj <ra...@microfocus.com> wrote:
>
> Hi,
>
> Currently we are using Flink 1.9.2 in our project and we are submitting Job
> from jar.
> below is that code we are using currently for jobsubmission:
>
> clusterClient.setDetached(true);
> clusterClient.run(packagedProgram,
> Integer.valueOf(params.get(Constants.PARAMS_PARALLELISM)));
>
> After upgrading it to Flink 1.11.1, above piece of code is not working.
> Also new packagedProgram constructor is expecting "Configuration" as
> parameter.
>
> How can we change that to work for our needs?
>
>
> Thanks,
>
>
>
> --
> Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/