You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@geode.apache.org by Jacob Barrett <ja...@vmware.com> on 2020/06/25 19:26:41 UTC
Docker on Windows
> On Jun 25, 2020, at 12:23 PM, Jens Deppe <jd...@vmware.com> wrote:
> It's been a couple of years since Sai and I tried (but failed) to dockerize the tests. I'm sure docker support has improved and it might be worth trying that again.
Docker on windows has improved a lot but wasn’t the major issue the docker plugin for Gradle needed some serious work?
I have recently been experimenting with the Docker/Kubernetes for Windows experience. Perhaps we can take another stab at this issue.
-Jake
Re: Docker on Windows
Posted by Jacob Barrett <ja...@vmware.com>.
Rebase complete and draft PR is open. I see the same null pointer issue Dale reported. Its a problem with the crufty Gradle Docker plugin we are using. It is the very reason we were thinking about making this move towards JUnit 5.
Here is what I think for next steps.
1) Disable test parallelization via Gradle Docker plugin.
2) Stabilize any other test issues.
3) Migrate to new Gradle runner for parallelization.
-Jake
On Jun 29, 2020, at 11:06 AM, Dale Emery <de...@vmware.com>> wrote:
• CI failures on JDK 11
• NPE thrown apparently by Gradle
Re: Docker on Windows
Posted by Jacob Barrett <ja...@vmware.com>.
On Jun 29, 2020, at 11:20 AM, Robert Houghton <rh...@vmware.com>> wrote:
@Jacob Barrett<ma...@vmware.com> @Dale Emery<ma...@vmware.com> I am excited to see progress on this. What I do not know, is what Junit5 buys us in terms of test isolation and parallelism compared to what we have now.
Based on Jen’s comment regarding fixing the docker plugin we use to launch JUnit tests in isolation.
On Jun 26, 2020, at 11:13 AM, Jens Deppe <jd...@vmware.com>> wrote:
A bigger effort (but I think more correct and sustainable) would be to switch to junit 5 where something like this could more easily be implemented.
Re: Docker on Windows
Posted by Robert Houghton <rh...@vmware.com>.
@Jacob Barrett<ma...@vmware.com> @Dale Emery<ma...@vmware.com> I am excited to see progress on this. What I do not know, is what Junit5 buys us in terms of test isolation and parallelism compared to what we have now.
From: Jacob Barrett <ja...@vmware.com>
Date: Monday, June 29, 2020 at 11:16 AM
To: dev@geode.apache.org <de...@geode.apache.org>
Subject: Re: Docker on Windows
Awesome! I will take a quick stab at rebasing the changes and see if anything has improved.
I will figure out a good place to setup a shared branch for collaboration.
-Jake
On Jun 29, 2020, at 11:06 AM, Dale Emery <de...@vmware.com>> wrote:
Here are my notes from my most recent attempts:
• November
• Added JUnit 5 to geode-junit API
• Configured geode-junit and geode-core to run all tests via JUnit 5
• CI failures on JDK 11
• NPE thrown apparently by Gradle
• December
• Ran tests in JDK 11 on my Mac
• Failures do not occur on my Mac
• Set gradle’s log level to info
• Results in CI logs so enormous that Chrome cannot display them in a bearable timeframe
• Then the whole PR CI pipeline became unusable
• So I was able to gain no further information about the cause of the gradle NPEs
Here is the branch the way I left things: https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fdemery-pivotal%2Fgeode%2Ftree%2Fold%2Fgeode-junit5&data=02%7C01%7Crhoughton%40vmware.com%7C09249e9ba58844c7563308d81c588743%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C1%7C637290513849574069&sdata=fPzImBwZ2zivCSdP6mPCGBFQNrpaahQuhZNpP3I4%2BIY%3D&reserved=0
About the PR CI pipeline becoming unusable… I don’t have any notes about what “unusable” means. I suspect it was unrelated to my PR, but I’m not sure.
Cheers,
Dale
On Jun 29, 2020, at 10:51 AM, Jacob Barrett <ja...@vmware.com>> wrote:
Dale,
Sorry I thought it was Kirk. Do you have a branch somewhere with your work so far? Can you refresh us all on the issues you hit and what you think the next steps would be?
Thanks,
Jake
On Jun 29, 2020, at 10:23 AM, Kirk Lund <kl...@apache.org>> wrote:
It was Dale who worked on migrating Geode to use JUnit 5. I know he ran
into some issues but I don't recall what they were. I'm definitely up for
helping on the JUnit 5 front!
On Fri, Jun 26, 2020 at 11:30 AM Jacob Barrett <ja...@vmware.com>> wrote:
If the effort to do both is less than the sum of each individually then I
say lets do it.
Kirk, I recall you putting some effort into JUnit 5 at some point.
-Jake
On Jun 26, 2020, at 11:13 AM, Jens Deppe <jd...@vmware.com>> wrote:
A bigger effort (but I think more correct and sustainable) would be to
switch to junit 5 where something like this could more easily be
implemented.
--Jens
On 6/26/20, 9:11 AM, "Robert Houghton" <rh...@vmware.com>> wrote:
The plugin code that spawns junit test workers on containers needs
some serious help. Aside from the benefit we would get on windows, we also
are blocked on getting to the next major version of Gradle with the current
tool. I really think it might be easier to write our own Gradle plugin at
this point.
From: Jacob Barrett <ja...@vmware.com>>
Date: Thursday, June 25, 2020 at 12:26 PM
To: dev@geode.apache.org<ma...@geode.apache.org> <de...@geode.apache.org>>
Subject: Docker on Windows
On Jun 25, 2020, at 12:23 PM, Jens Deppe <jd...@vmware.com>> wrote:
It's been a couple of years since Sai and I tried (but failed) to
dockerize the tests. I'm sure docker support has improved and it might be
worth trying that again.
Docker on windows has improved a lot but wasn’t the major issue the
docker plugin for Gradle needed some serious work?
I have recently been experimenting with the Docker/Kubernetes for
Windows experience. Perhaps we can take another stab at this issue.
-Jake
—
Dale Emery
demery@vmware.com<ma...@vmware.com>>
Re: Docker on Windows
Posted by Jacob Barrett <ja...@vmware.com>.
Awesome! I will take a quick stab at rebasing the changes and see if anything has improved.
I will figure out a good place to setup a shared branch for collaboration.
-Jake
On Jun 29, 2020, at 11:06 AM, Dale Emery <de...@vmware.com>> wrote:
Here are my notes from my most recent attempts:
• November
• Added JUnit 5 to geode-junit API
• Configured geode-junit and geode-core to run all tests via JUnit 5
• CI failures on JDK 11
• NPE thrown apparently by Gradle
• December
• Ran tests in JDK 11 on my Mac
• Failures do not occur on my Mac
• Set gradle’s log level to info
• Results in CI logs so enormous that Chrome cannot display them in a bearable timeframe
• Then the whole PR CI pipeline became unusable
• So I was able to gain no further information about the cause of the gradle NPEs
Here is the branch the way I left things: https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fdemery-pivotal%2Fgeode%2Ftree%2Fold%2Fgeode-junit5&data=02%7C01%7Cjabarrett%40vmware.com%7C1d35ce2b861a4caf5a4f08d81c574bc4%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637290508574727794&sdata=jTQbw4pKbcxwnUJ%2B3RC%2BvUOiyqHU5Xtro8Xg0D4vnFo%3D&reserved=0
About the PR CI pipeline becoming unusable… I don’t have any notes about what “unusable” means. I suspect it was unrelated to my PR, but I’m not sure.
Cheers,
Dale
On Jun 29, 2020, at 10:51 AM, Jacob Barrett <ja...@vmware.com>> wrote:
Dale,
Sorry I thought it was Kirk. Do you have a branch somewhere with your work so far? Can you refresh us all on the issues you hit and what you think the next steps would be?
Thanks,
Jake
On Jun 29, 2020, at 10:23 AM, Kirk Lund <kl...@apache.org>> wrote:
It was Dale who worked on migrating Geode to use JUnit 5. I know he ran
into some issues but I don't recall what they were. I'm definitely up for
helping on the JUnit 5 front!
On Fri, Jun 26, 2020 at 11:30 AM Jacob Barrett <ja...@vmware.com>> wrote:
If the effort to do both is less than the sum of each individually then I
say lets do it.
Kirk, I recall you putting some effort into JUnit 5 at some point.
-Jake
On Jun 26, 2020, at 11:13 AM, Jens Deppe <jd...@vmware.com>> wrote:
A bigger effort (but I think more correct and sustainable) would be to
switch to junit 5 where something like this could more easily be
implemented.
--Jens
On 6/26/20, 9:11 AM, "Robert Houghton" <rh...@vmware.com>> wrote:
The plugin code that spawns junit test workers on containers needs
some serious help. Aside from the benefit we would get on windows, we also
are blocked on getting to the next major version of Gradle with the current
tool. I really think it might be easier to write our own Gradle plugin at
this point.
From: Jacob Barrett <ja...@vmware.com>>
Date: Thursday, June 25, 2020 at 12:26 PM
To: dev@geode.apache.org<ma...@geode.apache.org> <de...@geode.apache.org>>
Subject: Docker on Windows
On Jun 25, 2020, at 12:23 PM, Jens Deppe <jd...@vmware.com>> wrote:
It's been a couple of years since Sai and I tried (but failed) to
dockerize the tests. I'm sure docker support has improved and it might be
worth trying that again.
Docker on windows has improved a lot but wasn’t the major issue the
docker plugin for Gradle needed some serious work?
I have recently been experimenting with the Docker/Kubernetes for
Windows experience. Perhaps we can take another stab at this issue.
-Jake
—
Dale Emery
demery@vmware.com<ma...@vmware.com>
Re: Docker on Windows
Posted by Dale Emery <de...@vmware.com>.
Here are my notes from my most recent attempts:
• November
• Added JUnit 5 to geode-junit API
• Configured geode-junit and geode-core to run all tests via JUnit 5
• CI failures on JDK 11
• NPE thrown apparently by Gradle
• December
• Ran tests in JDK 11 on my Mac
• Failures do not occur on my Mac
• Set gradle’s log level to info
• Results in CI logs so enormous that Chrome cannot display them in a bearable timeframe
• Then the whole PR CI pipeline became unusable
• So I was able to gain no further information about the cause of the gradle NPEs
Here is the branch the way I left things: https://github.com/demery-pivotal/geode/tree/old/geode-junit5
About the PR CI pipeline becoming unusable… I don’t have any notes about what “unusable” means. I suspect it was unrelated to my PR, but I’m not sure.
Cheers,
Dale
On Jun 29, 2020, at 10:51 AM, Jacob Barrett <ja...@vmware.com>> wrote:
Dale,
Sorry I thought it was Kirk. Do you have a branch somewhere with your work so far? Can you refresh us all on the issues you hit and what you think the next steps would be?
Thanks,
Jake
On Jun 29, 2020, at 10:23 AM, Kirk Lund <kl...@apache.org>> wrote:
It was Dale who worked on migrating Geode to use JUnit 5. I know he ran
into some issues but I don't recall what they were. I'm definitely up for
helping on the JUnit 5 front!
On Fri, Jun 26, 2020 at 11:30 AM Jacob Barrett <ja...@vmware.com>> wrote:
If the effort to do both is less than the sum of each individually then I
say lets do it.
Kirk, I recall you putting some effort into JUnit 5 at some point.
-Jake
On Jun 26, 2020, at 11:13 AM, Jens Deppe <jd...@vmware.com>> wrote:
A bigger effort (but I think more correct and sustainable) would be to
switch to junit 5 where something like this could more easily be
implemented.
--Jens
On 6/26/20, 9:11 AM, "Robert Houghton" <rh...@vmware.com>> wrote:
The plugin code that spawns junit test workers on containers needs
some serious help. Aside from the benefit we would get on windows, we also
are blocked on getting to the next major version of Gradle with the current
tool. I really think it might be easier to write our own Gradle plugin at
this point.
From: Jacob Barrett <ja...@vmware.com>>
Date: Thursday, June 25, 2020 at 12:26 PM
To: dev@geode.apache.org<ma...@geode.apache.org> <de...@geode.apache.org>>
Subject: Docker on Windows
On Jun 25, 2020, at 12:23 PM, Jens Deppe <jd...@vmware.com>> wrote:
It's been a couple of years since Sai and I tried (but failed) to
dockerize the tests. I'm sure docker support has improved and it might be
worth trying that again.
Docker on windows has improved a lot but wasn’t the major issue the
docker plugin for Gradle needed some serious work?
I have recently been experimenting with the Docker/Kubernetes for
Windows experience. Perhaps we can take another stab at this issue.
-Jake
—
Dale Emery
demery@vmware.com<ma...@vmware.com>
Re: Docker on Windows
Posted by Jacob Barrett <ja...@vmware.com>.
Dale,
Sorry I thought it was Kirk. Do you have a branch somewhere with your work so far? Can you refresh us all on the issues you hit and what you think the next steps would be?
Thanks,
Jake
> On Jun 29, 2020, at 10:23 AM, Kirk Lund <kl...@apache.org> wrote:
>
> It was Dale who worked on migrating Geode to use JUnit 5. I know he ran
> into some issues but I don't recall what they were. I'm definitely up for
> helping on the JUnit 5 front!
>
> On Fri, Jun 26, 2020 at 11:30 AM Jacob Barrett <ja...@vmware.com> wrote:
>
>> If the effort to do both is less than the sum of each individually then I
>> say lets do it.
>>
>> Kirk, I recall you putting some effort into JUnit 5 at some point.
>>
>> -Jake
>>
>>
>>> On Jun 26, 2020, at 11:13 AM, Jens Deppe <jd...@vmware.com> wrote:
>>>
>>> A bigger effort (but I think more correct and sustainable) would be to
>> switch to junit 5 where something like this could more easily be
>> implemented.
>>>
>>> --Jens
>>>
>>> On 6/26/20, 9:11 AM, "Robert Houghton" <rh...@vmware.com> wrote:
>>>
>>> The plugin code that spawns junit test workers on containers needs
>> some serious help. Aside from the benefit we would get on windows, we also
>> are blocked on getting to the next major version of Gradle with the current
>> tool. I really think it might be easier to write our own Gradle plugin at
>> this point.
>>>
>>>
>>> From: Jacob Barrett <ja...@vmware.com>
>>> Date: Thursday, June 25, 2020 at 12:26 PM
>>> To: dev@geode.apache.org <de...@geode.apache.org>
>>> Subject: Docker on Windows
>>>
>>>> On Jun 25, 2020, at 12:23 PM, Jens Deppe <jd...@vmware.com> wrote:
>>>> It's been a couple of years since Sai and I tried (but failed) to
>> dockerize the tests. I'm sure docker support has improved and it might be
>> worth trying that again.
>>>
>>> Docker on windows has improved a lot but wasn’t the major issue the
>> docker plugin for Gradle needed some serious work?
>>>
>>> I have recently been experimenting with the Docker/Kubernetes for
>> Windows experience. Perhaps we can take another stab at this issue.
>>>
>>> -Jake
>>>
>>
>>
Re: Docker on Windows
Posted by Kirk Lund <kl...@apache.org>.
It was Dale who worked on migrating Geode to use JUnit 5. I know he ran
into some issues but I don't recall what they were. I'm definitely up for
helping on the JUnit 5 front!
On Fri, Jun 26, 2020 at 11:30 AM Jacob Barrett <ja...@vmware.com> wrote:
> If the effort to do both is less than the sum of each individually then I
> say lets do it.
>
> Kirk, I recall you putting some effort into JUnit 5 at some point.
>
> -Jake
>
>
> > On Jun 26, 2020, at 11:13 AM, Jens Deppe <jd...@vmware.com> wrote:
> >
> > A bigger effort (but I think more correct and sustainable) would be to
> switch to junit 5 where something like this could more easily be
> implemented.
> >
> > --Jens
> >
> > On 6/26/20, 9:11 AM, "Robert Houghton" <rh...@vmware.com> wrote:
> >
> > The plugin code that spawns junit test workers on containers needs
> some serious help. Aside from the benefit we would get on windows, we also
> are blocked on getting to the next major version of Gradle with the current
> tool. I really think it might be easier to write our own Gradle plugin at
> this point.
> >
> >
> > From: Jacob Barrett <ja...@vmware.com>
> > Date: Thursday, June 25, 2020 at 12:26 PM
> > To: dev@geode.apache.org <de...@geode.apache.org>
> > Subject: Docker on Windows
> >
> >> On Jun 25, 2020, at 12:23 PM, Jens Deppe <jd...@vmware.com> wrote:
> >> It's been a couple of years since Sai and I tried (but failed) to
> dockerize the tests. I'm sure docker support has improved and it might be
> worth trying that again.
> >
> > Docker on windows has improved a lot but wasn’t the major issue the
> docker plugin for Gradle needed some serious work?
> >
> > I have recently been experimenting with the Docker/Kubernetes for
> Windows experience. Perhaps we can take another stab at this issue.
> >
> > -Jake
> >
>
>
Re: Docker on Windows
Posted by Jacob Barrett <ja...@vmware.com>.
If the effort to do both is less than the sum of each individually then I say lets do it.
Kirk, I recall you putting some effort into JUnit 5 at some point.
-Jake
> On Jun 26, 2020, at 11:13 AM, Jens Deppe <jd...@vmware.com> wrote:
>
> A bigger effort (but I think more correct and sustainable) would be to switch to junit 5 where something like this could more easily be implemented.
>
> --Jens
>
> On 6/26/20, 9:11 AM, "Robert Houghton" <rh...@vmware.com> wrote:
>
> The plugin code that spawns junit test workers on containers needs some serious help. Aside from the benefit we would get on windows, we also are blocked on getting to the next major version of Gradle with the current tool. I really think it might be easier to write our own Gradle plugin at this point.
>
>
> From: Jacob Barrett <ja...@vmware.com>
> Date: Thursday, June 25, 2020 at 12:26 PM
> To: dev@geode.apache.org <de...@geode.apache.org>
> Subject: Docker on Windows
>
>> On Jun 25, 2020, at 12:23 PM, Jens Deppe <jd...@vmware.com> wrote:
>> It's been a couple of years since Sai and I tried (but failed) to dockerize the tests. I'm sure docker support has improved and it might be worth trying that again.
>
> Docker on windows has improved a lot but wasn’t the major issue the docker plugin for Gradle needed some serious work?
>
> I have recently been experimenting with the Docker/Kubernetes for Windows experience. Perhaps we can take another stab at this issue.
>
> -Jake
>
Re: Docker on Windows
Posted by Jens Deppe <jd...@vmware.com>.
A bigger effort (but I think more correct and sustainable) would be to switch to junit 5 where something like this could more easily be implemented.
--Jens
On 6/26/20, 9:11 AM, "Robert Houghton" <rh...@vmware.com> wrote:
The plugin code that spawns junit test workers on containers needs some serious help. Aside from the benefit we would get on windows, we also are blocked on getting to the next major version of Gradle with the current tool. I really think it might be easier to write our own Gradle plugin at this point.
From: Jacob Barrett <ja...@vmware.com>
Date: Thursday, June 25, 2020 at 12:26 PM
To: dev@geode.apache.org <de...@geode.apache.org>
Subject: Docker on Windows
> On Jun 25, 2020, at 12:23 PM, Jens Deppe <jd...@vmware.com> wrote:
> It's been a couple of years since Sai and I tried (but failed) to dockerize the tests. I'm sure docker support has improved and it might be worth trying that again.
Docker on windows has improved a lot but wasn’t the major issue the docker plugin for Gradle needed some serious work?
I have recently been experimenting with the Docker/Kubernetes for Windows experience. Perhaps we can take another stab at this issue.
-Jake
Re: Docker on Windows
Posted by Robert Houghton <rh...@vmware.com>.
The plugin code that spawns junit test workers on containers needs some serious help. Aside from the benefit we would get on windows, we also are blocked on getting to the next major version of Gradle with the current tool. I really think it might be easier to write our own Gradle plugin at this point.
From: Jacob Barrett <ja...@vmware.com>
Date: Thursday, June 25, 2020 at 12:26 PM
To: dev@geode.apache.org <de...@geode.apache.org>
Subject: Docker on Windows
> On Jun 25, 2020, at 12:23 PM, Jens Deppe <jd...@vmware.com> wrote:
> It's been a couple of years since Sai and I tried (but failed) to dockerize the tests. I'm sure docker support has improved and it might be worth trying that again.
Docker on windows has improved a lot but wasn’t the major issue the docker plugin for Gradle needed some serious work?
I have recently been experimenting with the Docker/Kubernetes for Windows experience. Perhaps we can take another stab at this issue.
-Jake