You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by shane knapp ☠ <sk...@berkeley.edu> on 2021/04/14 17:20:31 UTC

[SPARK-34738] issues w/k8s+minikube and PV tests

please see:  https://issues.apache.org/jira/browse/SPARK-34738

i could really use a hand.  all k8s integration tests are currently broken,
and i'd rather spend the time fixing the latest version of minikube, k8s
and the docker virtualization layer than debug the 'old' way which uses the
kvm2/qemu virtualization layer.

thanks in advance,

shane
-- 
Shane Knapp
Computer Guy / Voice of Reason
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu

Re: [SPARK-34738] issues w/k8s+minikube and PV tests

Posted by shane knapp ☠ <sk...@berkeley.edu>.
alright, my canary build w/skipping the PV integration test passed w/the
docker driver:
https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-k8s-clone/20/

i'll put together a PR for this over the weekend (it's a one-liner) and
once we merge i can get the remaining workers upgraded early next week.

On Thu, Apr 15, 2021 at 3:05 PM shane knapp ☠ <sk...@berkeley.edu> wrote:

> i'm all for that...  and once they're turned off, we can finish the
> minikube/k8s/move-to-docker project in a couple of hours max.
>
> On Thu, Apr 15, 2021 at 3:00 PM Holden Karau <ho...@pigscanfly.ca> wrote:
>
>> What about if we just turn off the PV tests for now?
>> I'd be happy to help with the debugging/upgrading.
>>
>> On Thu, Apr 15, 2021 at 2:28 AM Rob Vesse <rv...@dotnetrdf.org> wrote:
>> >
>> > There’s at least one test (the persistent volumes one) that relies on
>> some Minikube functionality because we run integration tests for our
>> $dayjob Spark image builds using Docker for Desktop instead and that one
>> test fails because it relies on some minikube specific functionality.  That
>> test could be refactored because I think it’s just adding a minimal Ceph
>> cluster to the K8S cluster which can be done to any K8S cluster in principal
>> >
>> >
>> >
>> > Rob
>> >
>> >
>> >
>> > From: shane knapp ☠ <sk...@berkeley.edu>
>> > Date: Wednesday, 14 April 2021 at 18:56
>> > To: Frank Luo <lu...@gmail.com>
>> > Cc: dev <de...@spark.apache.org>, Brian K Shiratsuki <bk...@berkeley.edu>
>> > Subject: Re: [SPARK-34738] issues w/k8s+minikube and PV tests
>> >
>> >
>> >
>> > On Wed, Apr 14, 2021 at 10:32 AM Frank Luo <lu...@gmail.com> wrote:
>> >
>> > Is there any hard dependency on minkube? (i.e, GPU setting), kind (
>> https://kind.sigs.k8s.io/) is a stabler and simpler k8s cluster env on a
>> single machine (only requires docker) , it been widely used by k8s projects
>> testing.
>> >
>> >
>> >
>> > there are no hard deps on minikube...  it installs happily and
>> successfully runs every integration test except for persistent volumes.
>> >
>> >
>> >
>> > i haven't tried kind yet, but my time is super limited on this and i'd
>> rather not venture down another rabbit hole unless we absolutely have to.
>> >
>> >
>>
>>
>>
>> --
>> Twitter: https://twitter.com/holdenkarau
>> Books (Learning Spark, High Performance Spark, etc.):
>> https://amzn.to/2MaRAG9
>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>
>> ---------------------------------------------------------------------
>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>
>>
>
> --
> Shane Knapp
> Computer Guy / Voice of Reason
> UC Berkeley EECS Research / RISELab Staff Technical Lead
> https://rise.cs.berkeley.edu
>


-- 
Shane Knapp
Computer Guy / Voice of Reason
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu

Re: [SPARK-34738] issues w/k8s+minikube and PV tests

Posted by shane knapp ☠ <sk...@berkeley.edu>.
i'm all for that...  and once they're turned off, we can finish the
minikube/k8s/move-to-docker project in a couple of hours max.

On Thu, Apr 15, 2021 at 3:00 PM Holden Karau <ho...@pigscanfly.ca> wrote:

> What about if we just turn off the PV tests for now?
> I'd be happy to help with the debugging/upgrading.
>
> On Thu, Apr 15, 2021 at 2:28 AM Rob Vesse <rv...@dotnetrdf.org> wrote:
> >
> > There’s at least one test (the persistent volumes one) that relies on
> some Minikube functionality because we run integration tests for our
> $dayjob Spark image builds using Docker for Desktop instead and that one
> test fails because it relies on some minikube specific functionality.  That
> test could be refactored because I think it’s just adding a minimal Ceph
> cluster to the K8S cluster which can be done to any K8S cluster in principal
> >
> >
> >
> > Rob
> >
> >
> >
> > From: shane knapp ☠ <sk...@berkeley.edu>
> > Date: Wednesday, 14 April 2021 at 18:56
> > To: Frank Luo <lu...@gmail.com>
> > Cc: dev <de...@spark.apache.org>, Brian K Shiratsuki <bk...@berkeley.edu>
> > Subject: Re: [SPARK-34738] issues w/k8s+minikube and PV tests
> >
> >
> >
> > On Wed, Apr 14, 2021 at 10:32 AM Frank Luo <lu...@gmail.com> wrote:
> >
> > Is there any hard dependency on minkube? (i.e, GPU setting), kind (
> https://kind.sigs.k8s.io/) is a stabler and simpler k8s cluster env on a
> single machine (only requires docker) , it been widely used by k8s projects
> testing.
> >
> >
> >
> > there are no hard deps on minikube...  it installs happily and
> successfully runs every integration test except for persistent volumes.
> >
> >
> >
> > i haven't tried kind yet, but my time is super limited on this and i'd
> rather not venture down another rabbit hole unless we absolutely have to.
> >
> >
>
>
>
> --
> Twitter: https://twitter.com/holdenkarau
> Books (Learning Spark, High Performance Spark, etc.):
> https://amzn.to/2MaRAG9
> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>
>

-- 
Shane Knapp
Computer Guy / Voice of Reason
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu

Re: [SPARK-34738] issues w/k8s+minikube and PV tests

Posted by Holden Karau <ho...@pigscanfly.ca>.
What about if we just turn off the PV tests for now?
I'd be happy to help with the debugging/upgrading.

On Thu, Apr 15, 2021 at 2:28 AM Rob Vesse <rv...@dotnetrdf.org> wrote:
>
> There’s at least one test (the persistent volumes one) that relies on some Minikube functionality because we run integration tests for our $dayjob Spark image builds using Docker for Desktop instead and that one test fails because it relies on some minikube specific functionality.  That test could be refactored because I think it’s just adding a minimal Ceph cluster to the K8S cluster which can be done to any K8S cluster in principal
>
>
>
> Rob
>
>
>
> From: shane knapp ☠ <sk...@berkeley.edu>
> Date: Wednesday, 14 April 2021 at 18:56
> To: Frank Luo <lu...@gmail.com>
> Cc: dev <de...@spark.apache.org>, Brian K Shiratsuki <bk...@berkeley.edu>
> Subject: Re: [SPARK-34738] issues w/k8s+minikube and PV tests
>
>
>
> On Wed, Apr 14, 2021 at 10:32 AM Frank Luo <lu...@gmail.com> wrote:
>
> Is there any hard dependency on minkube? (i.e, GPU setting), kind (https://kind.sigs.k8s.io/) is a stabler and simpler k8s cluster env on a single machine (only requires docker) , it been widely used by k8s projects testing.
>
>
>
> there are no hard deps on minikube...  it installs happily and successfully runs every integration test except for persistent volumes.
>
>
>
> i haven't tried kind yet, but my time is super limited on this and i'd rather not venture down another rabbit hole unless we absolutely have to.
>
>



-- 
Twitter: https://twitter.com/holdenkarau
Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9
YouTube Live Streams: https://www.youtube.com/user/holdenkarau

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org


Re: [SPARK-34738] issues w/k8s+minikube and PV tests

Posted by Rob Vesse <rv...@dotnetrdf.org>.
There’s at least one test (the persistent volumes one) that relies on some Minikube functionality because we run integration tests for our $dayjob Spark image builds using Docker for Desktop instead and that one test fails because it relies on some minikube specific functionality.  That test could be refactored because I think it’s just adding a minimal Ceph cluster to the K8S cluster which can be done to any K8S cluster in principal

 

Rob

 

From: shane knapp ☠ <sk...@berkeley.edu>
Date: Wednesday, 14 April 2021 at 18:56
To: Frank Luo <lu...@gmail.com>
Cc: dev <de...@spark.apache.org>, Brian K Shiratsuki <bk...@berkeley.edu>
Subject: Re: [SPARK-34738] issues w/k8s+minikube and PV tests

 

On Wed, Apr 14, 2021 at 10:32 AM Frank Luo <lu...@gmail.com> wrote:

Is there any hard dependency on minkube? (i.e, GPU setting), kind (https://kind.sigs.k8s.io/) is a stabler and simpler k8s cluster env on a single machine (only requires docker) , it been widely used by k8s projects testing.

 

there are no hard deps on minikube...  it installs happily and successfully runs every integration test except for persistent volumes.

 

i haven't tried kind yet, but my time is super limited on this and i'd rather not venture down another rabbit hole unless we absolutely have to.

 


Re: [SPARK-34738] issues w/k8s+minikube and PV tests

Posted by shane knapp ☠ <sk...@berkeley.edu>.
On Wed, Apr 14, 2021 at 10:32 AM Frank Luo <lu...@gmail.com> wrote:

> Is there any hard dependency on minkube? (i.e, GPU setting), kind (
> https://kind.sigs.k8s.io/) is a stabler and simpler k8s cluster env on a
> single machine (only requires docker) , it been widely used by k8s projects
> testing.
>
> there are no hard deps on minikube...  it installs happily and
successfully runs every integration test except for persistent volumes.

i haven't tried kind yet, but my time is super limited on this and i'd
rather not venture down another rabbit hole unless we absolutely have to.

Re: [SPARK-34738] issues w/k8s+minikube and PV tests

Posted by Frank Luo <lu...@gmail.com>.
Is there any hard dependency on minkube? (i.e, GPU setting), kind (
https://kind.sigs.k8s.io/) is a stabler and simpler k8s cluster env on a
single machine (only requires docker) , it been widely used by k8s projects
testing.

I am interested at looking into test code to understand this aspect, but if
someone have more contexts to share, much appreciated.

Thanks
Hui

On Wed, Apr 14, 2021, 10:20 AM shane knapp ☠ <sk...@berkeley.edu> wrote:

> please see:  https://issues.apache.org/jira/browse/SPARK-34738
>
> i could really use a hand.  all k8s integration tests are currently
> broken, and i'd rather spend the time fixing the latest version of
> minikube, k8s and the docker virtualization layer than debug the 'old' way
> which uses the kvm2/qemu virtualization layer.
>
> thanks in advance,
>
> shane
> --
> Shane Knapp
> Computer Guy / Voice of Reason
> UC Berkeley EECS Research / RISELab Staff Technical Lead
> https://rise.cs.berkeley.edu
>