You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@heron.apache.org by Josh Fischer <jo...@joshfischer.io> on 2021/10/06 16:18:09 UTC
New Release
Hi,
I will be creating a new release for Heron. Does anyone have any concerns
before I start?
- Josh
Re: New Release
Posted by Ning Wang <wa...@gmail.com>.
Yeah. Our release isn't very frequent, and this is an interesting feature
to have.
On Wed, Oct 6, 2021 at 10:42 AM Josh Fischer <jo...@joshfischer.io> wrote:
> Let's wait.
>
> On Wed, Oct 6, 2021 at 12:34 PM Saad Ur Rahman <sa...@gmail.com>
> wrote:
>
> > Hi Ning,
> >
> > Just a quick update on the #3707 Pod Templates. I am actively working on
> > the PR, and it is currently in the testing phase (Minikube deployments).
> My
> > ability to engage in deployment testing is CPU constrained. Nick is
> > engaging as he can when time permits. The documentation is not complete.
> > The TravisCI pipeline builds tend to be flaky on one particular test,
> which
> > is also failing on the documentation branch, but the entire set of test
> > suites always pass on my system.
> >
> > I do not have any issues with a new release being rolled out with or
> > without #3707. If there are currently enough changes, then it would make
> > sense to go ahead, but I would not want to rush out anything that is not
> > fully tested and cleared.
> >
> >
> > /Saad
> >
> > On 2021/10/06 16:27:58, Ning Wang <wa...@gmail.com> wrote:
> > > Cool. Thanks.
> > >
> > > I am wondering how is the k8s configMap PR? Do we want to wait for it
> to
> > be
> > > merged?
> > >
> > > On Wed, Oct 6, 2021 at 9:18 AM Josh Fischer <jo...@joshfischer.io>
> wrote:
> > >
> > > > Hi,
> > > >
> > > > I will be creating a new release for Heron. Does anyone have any
> > concerns
> > > > before I start?
> > > >
> > > > - Josh
> > > >
> > >
> >
>
Re: New Release
Posted by Ning Wang <wa...@gmail.com>.
Thanks!
Have a few comments. Nothing major.
On Tue, Oct 12, 2021 at 2:29 PM Saad Ur Rahman <sa...@gmail.com>
wrote:
> Hi everyone,
>
> I am happy to report that I have completed a potential RC (issue #3710)
> which resolves issue #3707. The documentation is complete and also
> available for review.
>
> PR: https://github.com/apache/incubator-heron/pull/3710
> Doc: https://github.com/apache/incubator-heron/pull/3717
>
> I am seeking feedback and support from anyone willing to test. I will be
> moving to expose methods for testing within the V1Controller and extending
> the test suite whilst testing and reviewing is completed.
>
> On 2021/10/06 18:08:16, Saad Ur Rahman <sa...@gmail.com> wrote:
> > Thank you, Josh.
> >
> > I am currently having a look through the code upstream from the
> Kubernetes scheduler to figure out where the topologies are submitted from.
> From what I have grasped thus far, there are two parts of the job
> submission process directly relevant to #3707:
> > 1. Topology definition submission to the topology manager.
> > 2. Pod configuration definitions submitted to the K8s cluster.
> >
> > In the event of a pod config submission issue there needs to be a
> rollback on the topology submission. I am attempting to locate this in the
> <scheduler-core/SubmitterMain> and <stmgr/SchedulerStateManagerAdaptor>. As
> soon as this is resolved I can focus on the documentation.
> >
> >
> > /Saad
> >
> > On 2021/10/06 17:42:10, Josh Fischer <jo...@joshfischer.io> wrote:
> > > Let's wait.
> > >
> > > On Wed, Oct 6, 2021 at 12:34 PM Saad Ur Rahman <
> saad.ur.rahman@gmail.com>
> > > wrote:
> > >
> > > > Hi Ning,
> > > >
> > > > Just a quick update on the #3707 Pod Templates. I am actively
> working on
> > > > the PR, and it is currently in the testing phase (Minikube
> deployments). My
> > > > ability to engage in deployment testing is CPU constrained. Nick is
> > > > engaging as he can when time permits. The documentation is not
> complete.
> > > > The TravisCI pipeline builds tend to be flaky on one particular
> test, which
> > > > is also failing on the documentation branch, but the entire set of
> test
> > > > suites always pass on my system.
> > > >
> > > > I do not have any issues with a new release being rolled out with or
> > > > without #3707. If there are currently enough changes, then it would
> make
> > > > sense to go ahead, but I would not want to rush out anything that is
> not
> > > > fully tested and cleared.
> > > >
> > > >
> > > > /Saad
> > > >
> > > > On 2021/10/06 16:27:58, Ning Wang <wa...@gmail.com> wrote:
> > > > > Cool. Thanks.
> > > > >
> > > > > I am wondering how is the k8s configMap PR? Do we want to wait for
> it to
> > > > be
> > > > > merged?
> > > > >
> > > > > On Wed, Oct 6, 2021 at 9:18 AM Josh Fischer <jo...@joshfischer.io>
> wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I will be creating a new release for Heron. Does anyone have any
> > > > concerns
> > > > > > before I start?
> > > > > >
> > > > > > - Josh
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: New Release
Posted by Saad Ur Rahman <sa...@gmail.com>.
Hi everyone,
I am happy to report that I have completed a potential RC (issue #3710) which resolves issue #3707. The documentation is complete and also available for review.
PR: https://github.com/apache/incubator-heron/pull/3710
Doc: https://github.com/apache/incubator-heron/pull/3717
I am seeking feedback and support from anyone willing to test. I will be moving to expose methods for testing within the V1Controller and extending the test suite whilst testing and reviewing is completed.
On 2021/10/06 18:08:16, Saad Ur Rahman <sa...@gmail.com> wrote:
> Thank you, Josh.
>
> I am currently having a look through the code upstream from the Kubernetes scheduler to figure out where the topologies are submitted from. From what I have grasped thus far, there are two parts of the job submission process directly relevant to #3707:
> 1. Topology definition submission to the topology manager.
> 2. Pod configuration definitions submitted to the K8s cluster.
>
> In the event of a pod config submission issue there needs to be a rollback on the topology submission. I am attempting to locate this in the <scheduler-core/SubmitterMain> and <stmgr/SchedulerStateManagerAdaptor>. As soon as this is resolved I can focus on the documentation.
>
>
> /Saad
>
> On 2021/10/06 17:42:10, Josh Fischer <jo...@joshfischer.io> wrote:
> > Let's wait.
> >
> > On Wed, Oct 6, 2021 at 12:34 PM Saad Ur Rahman <sa...@gmail.com>
> > wrote:
> >
> > > Hi Ning,
> > >
> > > Just a quick update on the #3707 Pod Templates. I am actively working on
> > > the PR, and it is currently in the testing phase (Minikube deployments). My
> > > ability to engage in deployment testing is CPU constrained. Nick is
> > > engaging as he can when time permits. The documentation is not complete.
> > > The TravisCI pipeline builds tend to be flaky on one particular test, which
> > > is also failing on the documentation branch, but the entire set of test
> > > suites always pass on my system.
> > >
> > > I do not have any issues with a new release being rolled out with or
> > > without #3707. If there are currently enough changes, then it would make
> > > sense to go ahead, but I would not want to rush out anything that is not
> > > fully tested and cleared.
> > >
> > >
> > > /Saad
> > >
> > > On 2021/10/06 16:27:58, Ning Wang <wa...@gmail.com> wrote:
> > > > Cool. Thanks.
> > > >
> > > > I am wondering how is the k8s configMap PR? Do we want to wait for it to
> > > be
> > > > merged?
> > > >
> > > > On Wed, Oct 6, 2021 at 9:18 AM Josh Fischer <jo...@joshfischer.io> wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I will be creating a new release for Heron. Does anyone have any
> > > concerns
> > > > > before I start?
> > > > >
> > > > > - Josh
> > > > >
> > > >
> > >
> >
>
Re: New Release
Posted by Saad Ur Rahman <sa...@gmail.com>.
Thank you, Josh.
I am currently having a look through the code upstream from the Kubernetes scheduler to figure out where the topologies are submitted from. From what I have grasped thus far, there are two parts of the job submission process directly relevant to #3707:
1. Topology definition submission to the topology manager.
2. Pod configuration definitions submitted to the K8s cluster.
In the event of a pod config submission issue there needs to be a rollback on the topology submission. I am attempting to locate this in the <scheduler-core/SubmitterMain> and <stmgr/SchedulerStateManagerAdaptor>. As soon as this is resolved I can focus on the documentation.
/Saad
On 2021/10/06 17:42:10, Josh Fischer <jo...@joshfischer.io> wrote:
> Let's wait.
>
> On Wed, Oct 6, 2021 at 12:34 PM Saad Ur Rahman <sa...@gmail.com>
> wrote:
>
> > Hi Ning,
> >
> > Just a quick update on the #3707 Pod Templates. I am actively working on
> > the PR, and it is currently in the testing phase (Minikube deployments). My
> > ability to engage in deployment testing is CPU constrained. Nick is
> > engaging as he can when time permits. The documentation is not complete.
> > The TravisCI pipeline builds tend to be flaky on one particular test, which
> > is also failing on the documentation branch, but the entire set of test
> > suites always pass on my system.
> >
> > I do not have any issues with a new release being rolled out with or
> > without #3707. If there are currently enough changes, then it would make
> > sense to go ahead, but I would not want to rush out anything that is not
> > fully tested and cleared.
> >
> >
> > /Saad
> >
> > On 2021/10/06 16:27:58, Ning Wang <wa...@gmail.com> wrote:
> > > Cool. Thanks.
> > >
> > > I am wondering how is the k8s configMap PR? Do we want to wait for it to
> > be
> > > merged?
> > >
> > > On Wed, Oct 6, 2021 at 9:18 AM Josh Fischer <jo...@joshfischer.io> wrote:
> > >
> > > > Hi,
> > > >
> > > > I will be creating a new release for Heron. Does anyone have any
> > concerns
> > > > before I start?
> > > >
> > > > - Josh
> > > >
> > >
> >
>
Re: New Release
Posted by Josh Fischer <jo...@joshfischer.io>.
Let's wait.
On Wed, Oct 6, 2021 at 12:34 PM Saad Ur Rahman <sa...@gmail.com>
wrote:
> Hi Ning,
>
> Just a quick update on the #3707 Pod Templates. I am actively working on
> the PR, and it is currently in the testing phase (Minikube deployments). My
> ability to engage in deployment testing is CPU constrained. Nick is
> engaging as he can when time permits. The documentation is not complete.
> The TravisCI pipeline builds tend to be flaky on one particular test, which
> is also failing on the documentation branch, but the entire set of test
> suites always pass on my system.
>
> I do not have any issues with a new release being rolled out with or
> without #3707. If there are currently enough changes, then it would make
> sense to go ahead, but I would not want to rush out anything that is not
> fully tested and cleared.
>
>
> /Saad
>
> On 2021/10/06 16:27:58, Ning Wang <wa...@gmail.com> wrote:
> > Cool. Thanks.
> >
> > I am wondering how is the k8s configMap PR? Do we want to wait for it to
> be
> > merged?
> >
> > On Wed, Oct 6, 2021 at 9:18 AM Josh Fischer <jo...@joshfischer.io> wrote:
> >
> > > Hi,
> > >
> > > I will be creating a new release for Heron. Does anyone have any
> concerns
> > > before I start?
> > >
> > > - Josh
> > >
> >
>
Re: New Release
Posted by Saad Ur Rahman <sa...@gmail.com>.
Hi Ning,
Just a quick update on the #3707 Pod Templates. I am actively working on the PR, and it is currently in the testing phase (Minikube deployments). My ability to engage in deployment testing is CPU constrained. Nick is engaging as he can when time permits. The documentation is not complete. The TravisCI pipeline builds tend to be flaky on one particular test, which is also failing on the documentation branch, but the entire set of test suites always pass on my system.
I do not have any issues with a new release being rolled out with or without #3707. If there are currently enough changes, then it would make sense to go ahead, but I would not want to rush out anything that is not fully tested and cleared.
/Saad
On 2021/10/06 16:27:58, Ning Wang <wa...@gmail.com> wrote:
> Cool. Thanks.
>
> I am wondering how is the k8s configMap PR? Do we want to wait for it to be
> merged?
>
> On Wed, Oct 6, 2021 at 9:18 AM Josh Fischer <jo...@joshfischer.io> wrote:
>
> > Hi,
> >
> > I will be creating a new release for Heron. Does anyone have any concerns
> > before I start?
> >
> > - Josh
> >
>
Re: New Release
Posted by Ning Wang <wa...@gmail.com>.
Cool. Thanks.
I am wondering how is the k8s configMap PR? Do we want to wait for it to be
merged?
On Wed, Oct 6, 2021 at 9:18 AM Josh Fischer <jo...@joshfischer.io> wrote:
> Hi,
>
> I will be creating a new release for Heron. Does anyone have any concerns
> before I start?
>
> - Josh
>
Re: New Release
Posted by Ning Wang <wa...@gmail.com>.
Cool. Thanks.
I am wondering how is the k8s configMap PR? Do we want to wait for it to be
merged?
On Wed, Oct 6, 2021 at 9:18 AM Josh Fischer <jo...@joshfischer.io> wrote:
> Hi,
>
> I will be creating a new release for Heron. Does anyone have any concerns
> before I start?
>
> - Josh
>