Re: Support priority of the Flink YARN application in Flink 1.9

[Fabian Hueske <fh...@gmail.com>]

Hi Boxiu,

This sounds like a good feature.

Please have a look at our contribution guidelines [1].
To propose a feature, you should open a Jira issue [2] and start a
discussion there.

Please note that the feature freeze for the Flink 1.9 release happened a
few weeks ago.
The community is currently working on fixing blockers and testing the
release and won't accept new features.
A new feature would go into the next release.

Best, Fabian

[1] https://flink.apache.org/contributing/contribute-code.html
[2] https://issues.apache.org/jira/projects/FLINK/summary

Am Mi., 31. Juli 2019 um 13:24 Uhr schrieb Xintong Song <
tonysong820@gmail.com>:

> Thanks for bringing this up, Boxiu.
>
>
> The problem make sense to me. For me the concern is should we limit the
> priorities to 1-9 or not.
>
>
> I think it would be good to open a jira issue and have the discussion
> there.
>
>
> Thank you~
>
> Xintong Song
>
>
>
> On Wed, Jul 31, 2019 at 12:22 PM tian boxiu <bo...@gmail.com> wrote:
>
>> To: user@flink.apache.org
>> Title: Support priority of the Flink YARN application in Flink 1.9
>>
>> Hello everyone,
>>
>> Many of our batch jobs have changed the execution engine from spark to
>> flink. The flink is deployed on the yarn cluster. In some scenarios,
>> high-priority core tasks need to be submitted on the preemptive queue.
>> Previously, when running in spark, users configured the spark.yarn.priority
>> to control the priority, ranging from 1 to 9.
>>
>> Currently, flink 1.9 does not support yarn priority submission. The
>> default priority of yarn submission jobs is 1. Based on this, we
>> implemented the submission of the priority range from 1 to 9(from low to
>> high priority). When the user does not configure the parameters, the
>> priority is still 1. After we push the new feature online in June, The
>> system has been tested to run properly.
>>
>> So I want to create an issue and push the improvement back to community.
>> Hope someone can review it, thanks~
>>
>> Boxiu
>>
>

Re: Support priority of the Flink YARN application in Flink 1.9

[tian boxiu <bo...@gmail.com>]

Hello everyone,

I have created a Jira issue in:
https://issues.apache.org/jira/browse/FLINK-13548. Thanks for reviewing it.

Thank you~

Boxiu

Fabian Hueske <fh...@gmail.com> 于2019年8月2日周五 下午4:09写道：

> Hi Boxiu,
>
> This sounds like a good feature.
>
> Please have a look at our contribution guidelines [1].
> To propose a feature, you should open a Jira issue [2] and start a
> discussion there.
>
> Please note that the feature freeze for the Flink 1.9 release happened a
> few weeks ago.
> The community is currently working on fixing blockers and testing the
> release and won't accept new features.
> A new feature would go into the next release.
>
> Best, Fabian
>
> [1] https://flink.apache.org/contributing/contribute-code.html
> [2] https://issues.apache.org/jira/projects/FLINK/summary
>
> Am Mi., 31. Juli 2019 um 13:24 Uhr schrieb Xintong Song <
> tonysong820@gmail.com>:
>
>> Thanks for bringing this up, Boxiu.
>>
>>
>> The problem make sense to me. For me the concern is should we limit the
>> priorities to 1-9 or not.
>>
>>
>> I think it would be good to open a jira issue and have the discussion
>> there.
>>
>>
>> Thank you~
>>
>> Xintong Song
>>
>>
>>
>> On Wed, Jul 31, 2019 at 12:22 PM tian boxiu <bo...@gmail.com>
>> wrote:
>>
>>> To: user@flink.apache.org
>>> Title: Support priority of the Flink YARN application in Flink 1.9
>>>
>>> Hello everyone,
>>>
>>> Many of our batch jobs have changed the execution engine from spark to
>>> flink. The flink is deployed on the yarn cluster. In some scenarios,
>>> high-priority core tasks need to be submitted on the preemptive queue.
>>> Previously, when running in spark, users configured the spark.yarn.priority
>>> to control the priority, ranging from 1 to 9.
>>>
>>> Currently, flink 1.9 does not support yarn priority submission. The
>>> default priority of yarn submission jobs is 1. Based on this, we
>>> implemented the submission of the priority range from 1 to 9(from low to
>>> high priority). When the user does not configure the parameters, the
>>> priority is still 1. After we push the new feature online in June, The
>>> system has been tested to run properly.
>>>
>>> So I want to create an issue and push the improvement back to community.
>>> Hope someone can review it, thanks~
>>>
>>> Boxiu
>>>
>>