You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by David Thomas <dt...@gmail.com> on 2014/02/17 17:41:04 UTC

Shared access to RDD

Is it possible for multiple users to access the same RDD, say, from their
respective spark-shells?

Re: Shared access to RDD

Posted by Koert Kuipers <ko...@tresata.com>.

it is possible to run multiple queries using a shared SparkContext (which
holds the shared RDD). however this is not easily available in spark-shell
i believe.

alternatively tachyon can be used to share (serialized) RDDs

On Mon, Feb 17, 2014 at 11:41 AM, David Thomas <dt...@gmail.com> wrote:

> Is it possible for multiple users to access the same RDD, say, from their
> respective spark-shells?
>

Re: Shared access to RDD

Posted by Mayur Rustagi <ma...@gmail.com>.

No not right now...
1. You can look at jobserver opensourced by Ooyala which allows multiple
codes access to same rdd through rest interface
2. You can ofcourse go with shark server and load the data as sql table for
access, but that may or may not work for you as the RDD structure is
predefined by shark.
Its a popular request I believe..
Regards
Mayur

Mayur Rustagi
Ph: +919632149971
h <https://twitter.com/mayur_rustagi>ttp://www.sigmoidanalytics.com
https://twitter.com/mayur_rustagi

On Mon, Feb 17, 2014 at 8:41 AM, David Thomas <dt...@gmail.com> wrote:

> Is it possible for multiple users to access the same RDD, say, from their
> respective spark-shells?
>