You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Kittu M <ki...@gmail.com> on 2018/04/04 11:45:49 UTC
Scala program to spark-submit on k8 cluster
Hi,
I’m looking for a Scala program to spark submit a Scala application (spark
2.3 job) on k8 cluster .
Any help would be much appreciated. Thanks
Re: Scala program to spark-submit on k8 cluster
Posted by Thodoris Zois <zo...@ics.forth.gr>.
If you are looking for a Spark scheduler that runs on top of Kubernetes then this is the way to go: https://github.com/apache/spark/blob/master/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala
You can also have a look here:
https://apache-spark-on-k8s.github.io/userdocs/running-on-kubernetes.html
- Thodoris
> On 6 Apr 2018, at 15:04, Kittu M <ki...@gmail.com> wrote:
>
> Yes Yinan I’m looking for a Scala program which submits a Spark job to a k8s cluster by running spark-submit programmatically
>
>> On Wednesday, April 4, 2018, Yinan Li <li...@gmail.com> wrote:
>> Hi Kittu,
>>
>> What do you mean by "a Scala program"? Do you mean a program that submits a Spark job to a k8s cluster by running spark-submit programmatically, or some example Scala application that is to run on the cluster?
>>
>>> On Wed, Apr 4, 2018 at 4:45 AM, Kittu M <ki...@gmail.com> wrote:
>>> Hi,
>>>
>>> I’m looking for a Scala program to spark submit a Scala application (spark 2.3 job) on k8 cluster .
>>>
>>> Any help would be much appreciated. Thanks
>>>
>>>
>>
Re: Scala program to spark-submit on k8 cluster
Posted by Kittu M <ki...@gmail.com>.
Yes Yinan I’m looking for a Scala program which submits a Spark job to a
k8s cluster by running spark-submit programmatically
On Wednesday, April 4, 2018, Yinan Li <li...@gmail.com> wrote:
> Hi Kittu,
>
> What do you mean by "a Scala program"? Do you mean a program that submits
> a Spark job to a k8s cluster by running spark-submit programmatically, or
> some example Scala application that is to run on the cluster?
>
> On Wed, Apr 4, 2018 at 4:45 AM, Kittu M <ki...@gmail.com> wrote:
>
>> Hi,
>>
>> I’m looking for a Scala program to spark submit a Scala application
>> (spark 2.3 job) on k8 cluster .
>>
>> Any help would be much appreciated. Thanks
>>
>>
>>
>
Re: Scala program to spark-submit on k8 cluster
Posted by purna pradeep <pu...@gmail.com>.
yes “REST application that submits a Spark job to a k8s cluster by running
spark-submit programmatically” and also would like to expose as a
Kubernetes service so that clients can access as any other Rest api
On Wed, Apr 4, 2018 at 12:25 PM Yinan Li <li...@gmail.com> wrote:
> Hi Kittu,
>
> What do you mean by "a Scala program"? Do you mean a program that submits
> a Spark job to a k8s cluster by running spark-submit programmatically, or
> some example Scala application that is to run on the cluster?
>
> On Wed, Apr 4, 2018 at 4:45 AM, Kittu M <ki...@gmail.com> wrote:
>
>> Hi,
>>
>> I’m looking for a Scala program to spark submit a Scala application
>> (spark 2.3 job) on k8 cluster .
>>
>> Any help would be much appreciated. Thanks
>>
>>
>>
>
Re: Scala program to spark-submit on k8 cluster
Posted by Yinan Li <li...@gmail.com>.
Hi Kittu,
What do you mean by "a Scala program"? Do you mean a program that submits a
Spark job to a k8s cluster by running spark-submit programmatically, or
some example Scala application that is to run on the cluster?
On Wed, Apr 4, 2018 at 4:45 AM, Kittu M <ki...@gmail.com> wrote:
> Hi,
>
> I’m looking for a Scala program to spark submit a Scala application (spark
> 2.3 job) on k8 cluster .
>
> Any help would be much appreciated. Thanks
>
>
>