You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@yunikorn.apache.org by Wilfred Spiegelenburg <wi...@apache.org> on 2023/07/28 02:41:18 UTC
[DISCUSS] removal of the yunikorn application CRD
We added the YuniKorn application CRD via YUNIKORN-170 [1]. The idea
was back then that we used the CRD to implement gang scheduling.
During the later design of gang scheduling we completely stepped away
from using the CRD as the basis for gang scheduling. Some of the other
advantages that we were expecting from the CRD were never observed
(finished state for an application, one point to manage pods). The
Spark CRD integration was mostly reversed [2] in favour of normal pod
handling due to issues.
The second phase for the application CRD YUNIKORN-599 [3] was never
started due to the limited advantages we expected to get.
There have been no changes in the code or jiras logged against the CRD
for two years besides making the build work on later K8s versions [4]
The current usage of the application CRD is limited to the
TaskGrooupDefinition being used for gang scheduling.
Based on all this I would like to start the discussion on removing the
application CRD from YuniKorn. Frank Yang has looked at the changes
needed to remove the CRD and the impact on the code for the K8shim. A
commit with all the changes can be seen in his repo [5] to reference.
The change will remove almost 3,000 lines of code just from the K8shim
repository. There will be some further changes needed to clean up the
build (K8shim) and helmchart (release). Those changes will be removal
of scripts and code only.
Before we progress with this further we would like to know:
* If the application CRD is used by anyone.
* If it is used, what part(s) of the CRD are used and what is it used for?
* Is removing the CRD in YuniKorn 1.4.0 OK or do we need to push to a
later release.
Objections, comments please let us know.
Thank you,
Wilfred
[1] https://issues.apache.org/jira/browse/YUNIKORN-170
[2] https://issues.apache.org/jira/browse/YUNIKORN-643
[3] https://issues.apache.org/jira/browse/YUNIKORN-599
[4] https://github.com/apache/yunikorn-k8shim/commits/master/pkg/apis/yunikorn.apache.org
[5] https://github.com/FrankYang0529/yunikorn-k8shim/commit/354248b3c24accd679ff2d5a557c599123a58408
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: dev-help@yunikorn.apache.org
Re: [DISCUSS] removal of the yunikorn application CRD
Posted by Wilfred Spiegelenburg <wi...@apache.org>.
Thank you for the feedback on the removal of the application CRD.
I will go ahead and log a jira and target that for YuniKorn 1.4.
Since Frank Yang has already done the POC for it I will assign the jira to him.
Thank you,
Wilfred
On Sat, 29 Jul 2023 at 16:09, Wilfred Spiegelenburg <wi...@apache.org> wrote:
>
> This is purely about the YuniKorn application CRD. Not about the Spark
> operator integration, we do not want to touch that one
> The Spark link was only to show that a CRD provides limited functionality.
>
> The POC that Frank did was also just the YuniKorn application CRD
>
> Wilfred
>
> On Sat, 29 Jul 2023 at 05:52, Weiwei Yang <ww...@apache.org> wrote:
> >
> > Remove YuniKorn application CRD should be fine, I don't think we ever
> > reached the GA for this feature, and I don't think anyone is using it.
> > The operator plugin, however, I don't think we should remove, for example
> > the Spark CRD plugin
> > <https://github.com/apache/yunikorn-k8shim/blob/master/pkg/appmgmt/sparkoperator/spark.go>
> > provides
> > the hook to YK in order to update the Spark job status more consistently.
> >
> > Weiwei
> >
> > On Fri, Jul 28, 2023 at 12:06 PM Craig Condit <cc...@apache.org> wrote:
> >
> > > I’m definitely in favor of removing the CRD, and sooner rather than later.
> > > It negatively impacts some of the ongoing refactoring tasks as it
> > > influences the recovery pipeline. I think given the alpha state of the CRD,
> > > if there are no objections we can remove this in 1.4.0.
> > >
> > > Craig
> > >
> > >
> > > > On Jul 27, 2023, at 9:41 PM, Wilfred Spiegelenburg <wi...@apache.org>
> > > wrote:
> > > >
> > > > We added the YuniKorn application CRD via YUNIKORN-170 [1]. The idea
> > > > was back then that we used the CRD to implement gang scheduling.
> > > > During the later design of gang scheduling we completely stepped away
> > > > from using the CRD as the basis for gang scheduling. Some of the other
> > > > advantages that we were expecting from the CRD were never observed
> > > > (finished state for an application, one point to manage pods). The
> > > > Spark CRD integration was mostly reversed [2] in favour of normal pod
> > > > handling due to issues.
> > > > The second phase for the application CRD YUNIKORN-599 [3] was never
> > > > started due to the limited advantages we expected to get.
> > > >
> > > > There have been no changes in the code or jiras logged against the CRD
> > > > for two years besides making the build work on later K8s versions [4]
> > > > The current usage of the application CRD is limited to the
> > > > TaskGrooupDefinition being used for gang scheduling.
> > > >
> > > > Based on all this I would like to start the discussion on removing the
> > > > application CRD from YuniKorn. Frank Yang has looked at the changes
> > > > needed to remove the CRD and the impact on the code for the K8shim. A
> > > > commit with all the changes can be seen in his repo [5] to reference.
> > > > The change will remove almost 3,000 lines of code just from the K8shim
> > > > repository. There will be some further changes needed to clean up the
> > > > build (K8shim) and helmchart (release). Those changes will be removal
> > > > of scripts and code only.
> > > >
> > > > Before we progress with this further we would like to know:
> > > > * If the application CRD is used by anyone.
> > > > * If it is used, what part(s) of the CRD are used and what is it used
> > > for?
> > > > * Is removing the CRD in YuniKorn 1.4.0 OK or do we need to push to a
> > > > later release.
> > > >
> > > > Objections, comments please let us know.
> > > >
> > > > Thank you,
> > > > Wilfred
> > > >
> > > > [1] https://issues.apache.org/jira/browse/YUNIKORN-170
> > > > [2] https://issues.apache.org/jira/browse/YUNIKORN-643
> > > > [3] https://issues.apache.org/jira/browse/YUNIKORN-599
> > > > [4]
> > > https://github.com/apache/yunikorn-k8shim/commits/master/pkg/apis/yunikorn.apache.org
> > > > [5]
> > > https://github.com/FrankYang0529/yunikorn-k8shim/commit/354248b3c24accd679ff2d5a557c599123a58408
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
> > > > For additional commands, e-mail: dev-help@yunikorn.apache.org
> > > >
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
> > > For additional commands, e-mail: dev-help@yunikorn.apache.org
> > >
> > >
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: dev-help@yunikorn.apache.org
Re: [DISCUSS] removal of the yunikorn application CRD
Posted by Wilfred Spiegelenburg <wi...@apache.org>.
This is purely about the YuniKorn application CRD. Not about the Spark
operator integration, we do not want to touch that one
The Spark link was only to show that a CRD provides limited functionality.
The POC that Frank did was also just the YuniKorn application CRD
Wilfred
On Sat, 29 Jul 2023 at 05:52, Weiwei Yang <ww...@apache.org> wrote:
>
> Remove YuniKorn application CRD should be fine, I don't think we ever
> reached the GA for this feature, and I don't think anyone is using it.
> The operator plugin, however, I don't think we should remove, for example
> the Spark CRD plugin
> <https://github.com/apache/yunikorn-k8shim/blob/master/pkg/appmgmt/sparkoperator/spark.go>
> provides
> the hook to YK in order to update the Spark job status more consistently.
>
> Weiwei
>
> On Fri, Jul 28, 2023 at 12:06 PM Craig Condit <cc...@apache.org> wrote:
>
> > I’m definitely in favor of removing the CRD, and sooner rather than later.
> > It negatively impacts some of the ongoing refactoring tasks as it
> > influences the recovery pipeline. I think given the alpha state of the CRD,
> > if there are no objections we can remove this in 1.4.0.
> >
> > Craig
> >
> >
> > > On Jul 27, 2023, at 9:41 PM, Wilfred Spiegelenburg <wi...@apache.org>
> > wrote:
> > >
> > > We added the YuniKorn application CRD via YUNIKORN-170 [1]. The idea
> > > was back then that we used the CRD to implement gang scheduling.
> > > During the later design of gang scheduling we completely stepped away
> > > from using the CRD as the basis for gang scheduling. Some of the other
> > > advantages that we were expecting from the CRD were never observed
> > > (finished state for an application, one point to manage pods). The
> > > Spark CRD integration was mostly reversed [2] in favour of normal pod
> > > handling due to issues.
> > > The second phase for the application CRD YUNIKORN-599 [3] was never
> > > started due to the limited advantages we expected to get.
> > >
> > > There have been no changes in the code or jiras logged against the CRD
> > > for two years besides making the build work on later K8s versions [4]
> > > The current usage of the application CRD is limited to the
> > > TaskGrooupDefinition being used for gang scheduling.
> > >
> > > Based on all this I would like to start the discussion on removing the
> > > application CRD from YuniKorn. Frank Yang has looked at the changes
> > > needed to remove the CRD and the impact on the code for the K8shim. A
> > > commit with all the changes can be seen in his repo [5] to reference.
> > > The change will remove almost 3,000 lines of code just from the K8shim
> > > repository. There will be some further changes needed to clean up the
> > > build (K8shim) and helmchart (release). Those changes will be removal
> > > of scripts and code only.
> > >
> > > Before we progress with this further we would like to know:
> > > * If the application CRD is used by anyone.
> > > * If it is used, what part(s) of the CRD are used and what is it used
> > for?
> > > * Is removing the CRD in YuniKorn 1.4.0 OK or do we need to push to a
> > > later release.
> > >
> > > Objections, comments please let us know.
> > >
> > > Thank you,
> > > Wilfred
> > >
> > > [1] https://issues.apache.org/jira/browse/YUNIKORN-170
> > > [2] https://issues.apache.org/jira/browse/YUNIKORN-643
> > > [3] https://issues.apache.org/jira/browse/YUNIKORN-599
> > > [4]
> > https://github.com/apache/yunikorn-k8shim/commits/master/pkg/apis/yunikorn.apache.org
> > > [5]
> > https://github.com/FrankYang0529/yunikorn-k8shim/commit/354248b3c24accd679ff2d5a557c599123a58408
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
> > > For additional commands, e-mail: dev-help@yunikorn.apache.org
> > >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
> > For additional commands, e-mail: dev-help@yunikorn.apache.org
> >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: dev-help@yunikorn.apache.org
Re: [DISCUSS] removal of the yunikorn application CRD
Posted by Weiwei Yang <ww...@apache.org>.
Remove YuniKorn application CRD should be fine, I don't think we ever
reached the GA for this feature, and I don't think anyone is using it.
The operator plugin, however, I don't think we should remove, for example
the Spark CRD plugin
<https://github.com/apache/yunikorn-k8shim/blob/master/pkg/appmgmt/sparkoperator/spark.go>
provides
the hook to YK in order to update the Spark job status more consistently.
Weiwei
On Fri, Jul 28, 2023 at 12:06 PM Craig Condit <cc...@apache.org> wrote:
> I’m definitely in favor of removing the CRD, and sooner rather than later.
> It negatively impacts some of the ongoing refactoring tasks as it
> influences the recovery pipeline. I think given the alpha state of the CRD,
> if there are no objections we can remove this in 1.4.0.
>
> Craig
>
>
> > On Jul 27, 2023, at 9:41 PM, Wilfred Spiegelenburg <wi...@apache.org>
> wrote:
> >
> > We added the YuniKorn application CRD via YUNIKORN-170 [1]. The idea
> > was back then that we used the CRD to implement gang scheduling.
> > During the later design of gang scheduling we completely stepped away
> > from using the CRD as the basis for gang scheduling. Some of the other
> > advantages that we were expecting from the CRD were never observed
> > (finished state for an application, one point to manage pods). The
> > Spark CRD integration was mostly reversed [2] in favour of normal pod
> > handling due to issues.
> > The second phase for the application CRD YUNIKORN-599 [3] was never
> > started due to the limited advantages we expected to get.
> >
> > There have been no changes in the code or jiras logged against the CRD
> > for two years besides making the build work on later K8s versions [4]
> > The current usage of the application CRD is limited to the
> > TaskGrooupDefinition being used for gang scheduling.
> >
> > Based on all this I would like to start the discussion on removing the
> > application CRD from YuniKorn. Frank Yang has looked at the changes
> > needed to remove the CRD and the impact on the code for the K8shim. A
> > commit with all the changes can be seen in his repo [5] to reference.
> > The change will remove almost 3,000 lines of code just from the K8shim
> > repository. There will be some further changes needed to clean up the
> > build (K8shim) and helmchart (release). Those changes will be removal
> > of scripts and code only.
> >
> > Before we progress with this further we would like to know:
> > * If the application CRD is used by anyone.
> > * If it is used, what part(s) of the CRD are used and what is it used
> for?
> > * Is removing the CRD in YuniKorn 1.4.0 OK or do we need to push to a
> > later release.
> >
> > Objections, comments please let us know.
> >
> > Thank you,
> > Wilfred
> >
> > [1] https://issues.apache.org/jira/browse/YUNIKORN-170
> > [2] https://issues.apache.org/jira/browse/YUNIKORN-643
> > [3] https://issues.apache.org/jira/browse/YUNIKORN-599
> > [4]
> https://github.com/apache/yunikorn-k8shim/commits/master/pkg/apis/yunikorn.apache.org
> > [5]
> https://github.com/FrankYang0529/yunikorn-k8shim/commit/354248b3c24accd679ff2d5a557c599123a58408
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
> > For additional commands, e-mail: dev-help@yunikorn.apache.org
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
> For additional commands, e-mail: dev-help@yunikorn.apache.org
>
>
Re: [DISCUSS] removal of the yunikorn application CRD
Posted by Craig Condit <cc...@apache.org>.
I’m definitely in favor of removing the CRD, and sooner rather than later. It negatively impacts some of the ongoing refactoring tasks as it influences the recovery pipeline. I think given the alpha state of the CRD, if there are no objections we can remove this in 1.4.0.
Craig
> On Jul 27, 2023, at 9:41 PM, Wilfred Spiegelenburg <wi...@apache.org> wrote:
>
> We added the YuniKorn application CRD via YUNIKORN-170 [1]. The idea
> was back then that we used the CRD to implement gang scheduling.
> During the later design of gang scheduling we completely stepped away
> from using the CRD as the basis for gang scheduling. Some of the other
> advantages that we were expecting from the CRD were never observed
> (finished state for an application, one point to manage pods). The
> Spark CRD integration was mostly reversed [2] in favour of normal pod
> handling due to issues.
> The second phase for the application CRD YUNIKORN-599 [3] was never
> started due to the limited advantages we expected to get.
>
> There have been no changes in the code or jiras logged against the CRD
> for two years besides making the build work on later K8s versions [4]
> The current usage of the application CRD is limited to the
> TaskGrooupDefinition being used for gang scheduling.
>
> Based on all this I would like to start the discussion on removing the
> application CRD from YuniKorn. Frank Yang has looked at the changes
> needed to remove the CRD and the impact on the code for the K8shim. A
> commit with all the changes can be seen in his repo [5] to reference.
> The change will remove almost 3,000 lines of code just from the K8shim
> repository. There will be some further changes needed to clean up the
> build (K8shim) and helmchart (release). Those changes will be removal
> of scripts and code only.
>
> Before we progress with this further we would like to know:
> * If the application CRD is used by anyone.
> * If it is used, what part(s) of the CRD are used and what is it used for?
> * Is removing the CRD in YuniKorn 1.4.0 OK or do we need to push to a
> later release.
>
> Objections, comments please let us know.
>
> Thank you,
> Wilfred
>
> [1] https://issues.apache.org/jira/browse/YUNIKORN-170
> [2] https://issues.apache.org/jira/browse/YUNIKORN-643
> [3] https://issues.apache.org/jira/browse/YUNIKORN-599
> [4] https://github.com/apache/yunikorn-k8shim/commits/master/pkg/apis/yunikorn.apache.org
> [5] https://github.com/FrankYang0529/yunikorn-k8shim/commit/354248b3c24accd679ff2d5a557c599123a58408
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
> For additional commands, e-mail: dev-help@yunikorn.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: dev-help@yunikorn.apache.org