You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Jakob Odersky <ja...@odersky.com> on 2016/01/27 19:39:21 UTC
Mutiple spark contexts
A while ago, I remember reading that multiple active Spark contexts
per JVM was a possible future enhancement.
I was wondering if this is still on the roadmap, what the major
obstacles are and if I can be of any help in adding this feature?
regards,
--Jakob
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org
Re: Mutiple spark contexts
Posted by Reynold Xin <rx...@databricks.com>.
There are no major obstacles, just a million tiny obstacles that would take
forever to fix.
On Wed, Jan 27, 2016 at 10:39 AM, Jakob Odersky <ja...@odersky.com> wrote:
> A while ago, I remember reading that multiple active Spark contexts
> per JVM was a possible future enhancement.
> I was wondering if this is still on the roadmap, what the major
> obstacles are and if I can be of any help in adding this feature?
>
> regards,
> --Jakob
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> For additional commands, e-mail: dev-help@spark.apache.org
>
>
Re: Mutiple spark contexts
Posted by Nicholas Chammas <ni...@gmail.com>.
There is a lengthy discussion about this on the JIRA:
https://issues.apache.org/jira/browse/SPARK-2243
On Wed, Jan 27, 2016 at 1:43 PM Herman van Hövell tot Westerflier <
hvanhovell@questtec.nl> wrote:
> Just out of curiousity. What is the use case for having multiple active
> contexts in a single JVM?
>
> Kind regards,
>
> Herman van Hövell
>
> 2016-01-27 19:41 GMT+01:00 Ashish Soni <as...@gmail.com>:
>
>> There is a property you need to set which is
>> spark.driver.allowMultipleContexts=true
>>
>> Ashish
>>
>> On Wed, Jan 27, 2016 at 1:39 PM, Jakob Odersky <ja...@odersky.com> wrote:
>>
>>> A while ago, I remember reading that multiple active Spark contexts
>>> per JVM was a possible future enhancement.
>>> I was wondering if this is still on the roadmap, what the major
>>> obstacles are and if I can be of any help in adding this feature?
>>>
>>> regards,
>>> --Jakob
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
>>> For additional commands, e-mail: dev-help@spark.apache.org
>>>
>>>
>>
>
Re: Mutiple spark contexts
Posted by Herman van Hövell tot Westerflier <hv...@questtec.nl>.
Just out of curiousity. What is the use case for having multiple active
contexts in a single JVM?
Kind regards,
Herman van Hövell
2016-01-27 19:41 GMT+01:00 Ashish Soni <as...@gmail.com>:
> There is a property you need to set which is
> spark.driver.allowMultipleContexts=true
>
> Ashish
>
> On Wed, Jan 27, 2016 at 1:39 PM, Jakob Odersky <ja...@odersky.com> wrote:
>
>> A while ago, I remember reading that multiple active Spark contexts
>> per JVM was a possible future enhancement.
>> I was wondering if this is still on the roadmap, what the major
>> obstacles are and if I can be of any help in adding this feature?
>>
>> regards,
>> --Jakob
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
>> For additional commands, e-mail: dev-help@spark.apache.org
>>
>>
>
Re: Mutiple spark contexts
Posted by Ashish Soni <as...@gmail.com>.
There is a property you need to set which is
spark.driver.allowMultipleContexts=true
Ashish
On Wed, Jan 27, 2016 at 1:39 PM, Jakob Odersky <ja...@odersky.com> wrote:
> A while ago, I remember reading that multiple active Spark contexts
> per JVM was a possible future enhancement.
> I was wondering if this is still on the roadmap, what the major
> obstacles are and if I can be of any help in adding this feature?
>
> regards,
> --Jakob
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> For additional commands, e-mail: dev-help@spark.apache.org
>
>