You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Dhaval Patel <dh...@gmail.com> on 2015/08/20 18:49:18 UTC
How to list all dataframes and RDDs available in current session?
Hi:
I have been working on few example using zeppelin.
I have been trying to find a command that would list all *dataframes/RDDs*
that has been created in current session. Anyone knows if there is any such
commands available?
Something similar to SparkSQL to list all temp tables :
show tables;
Thanks,
Dhaval
Re: How to list all dataframes and RDDs available in current session?
Posted by Dhaval Patel <dh...@gmail.com>.
Apologies!!!! I accidentally included the DL on BCC instead of in To:. The
actual email message is below:
===========================================================================
Hi:
I have been working on few example using zeppelin.
I have been trying to find a command that would list all *dataframes/RDDs*
that has been created in current session. Anyone knows if there is any such
commands available?
Something similar to SparkSQL to list all temp tables :
show tables;
Thanks,
Dhaval
>
> On Thu, Aug 20, 2015 at 12:49 PM, Dhaval Patel <dh...@gmail.com>
> wrote:
>
>> Hi:
>>
>> I have been working on few example using zeppelin.
>>
>> I have been trying to find a command that would list all
>> *dataframes/RDDs* that has been created in current session. Anyone knows if
>> there is any such commands available?
>>
>> Something similar to SparkSQL to list all temp tables :
>> show tables;
>>
>> Thanks,
>> Dhaval
>>
>
>
Re: How to list all dataframes and RDDs available in current session?
Posted by Dhaval Gmail <dh...@gmail.com>.
Okay but "how?" thats what I am trying to figure out 😀? Any command you would suggest?
"Sent from my iPhone, plaese excuse any typos :)"
> On Aug 21, 2015, at 11:45 PM, Raghavendra Pandey <ra...@gmail.com> wrote:
>
> You get the list of all the persistet rdd using spark context...
>> On Aug 21, 2015 12:06 AM, "Rishitesh Mishra" <ri...@gmail.com> wrote:
>> I am not sure if you can view all RDDs in a session. Tables are maintained in a catalogue . Hence its easier. However you can see the DAG representation , which lists all the RDDs in a job , with Spark UI.
>>
>>> On 20 Aug 2015 22:34, "Dhaval Patel" <dh...@gmail.com> wrote:
>>> Apologies!!!!
>>>
>>> I accidentally included Spark User DL on BCC. The actual email message is below.
>>> =============================================================
>>>
>>>
>>> Hi:
>>>
>>> I have been working on few example using zeppelin.
>>>
>>> I have been trying to find a command that would list all *dataframes/RDDs* that has been created in current session. Anyone knows if there is any such commands available?
>>>
>>> Something similar to SparkSQL to list all temp tables :
>>> show tables;
>>>
>>> Thanks,
>>> Dhaval
>>>
>>>
>>>
>>>> On Thu, Aug 20, 2015 at 12:49 PM, Dhaval Patel <dh...@gmail.com> wrote:
>>>> Hi:
>>>>
>>>> I have been working on few example using zeppelin.
>>>>
>>>> I have been trying to find a command that would list all *dataframes/RDDs* that has been created in current session. Anyone knows if there is any such commands available?
>>>>
>>>> Something similar to SparkSQL to list all temp tables :
>>>> show tables;
>>>>
>>>> Thanks,
>>>> Dhaval
Re: How to list all dataframes and RDDs available in current session?
Posted by Raghavendra Pandey <ra...@gmail.com>.
You get the list of all the persistet rdd using spark context...
On Aug 21, 2015 12:06 AM, "Rishitesh Mishra" <ri...@gmail.com>
wrote:
> I am not sure if you can view all RDDs in a session. Tables are maintained
> in a catalogue . Hence its easier. However you can see the DAG
> representation , which lists all the RDDs in a job , with Spark UI.
> On 20 Aug 2015 22:34, "Dhaval Patel" <dh...@gmail.com> wrote:
>
>> Apologies!!!!
>>
>> I accidentally included Spark User DL on BCC. The actual email message is
>> below.
>> =============================================================
>>
>>
>> Hi:
>>
>> I have been working on few example using zeppelin.
>>
>> I have been trying to find a command that would list all
>> *dataframes/RDDs* that has been created in current session. Anyone knows if
>> there is any such commands available?
>>
>> Something similar to SparkSQL to list all temp tables :
>> show tables;
>>
>> Thanks,
>> Dhaval
>>
>>
>>
>> On Thu, Aug 20, 2015 at 12:49 PM, Dhaval Patel <dh...@gmail.com>
>> wrote:
>>
>>> Hi:
>>>
>>> I have been working on few example using zeppelin.
>>>
>>> I have been trying to find a command that would list all
>>> *dataframes/RDDs* that has been created in current session. Anyone knows if
>>> there is any such commands available?
>>>
>>> Something similar to SparkSQL to list all temp tables :
>>> show tables;
>>>
>>> Thanks,
>>> Dhaval
>>>
>>
>>
Re: How to list all dataframes and RDDs available in current session?
Posted by Rishitesh Mishra <ri...@gmail.com>.
I am not sure if you can view all RDDs in a session. Tables are maintained
in a catalogue . Hence its easier. However you can see the DAG
representation , which lists all the RDDs in a job , with Spark UI.
On 20 Aug 2015 22:34, "Dhaval Patel" <dh...@gmail.com> wrote:
> Apologies!!!!
>
> I accidentally included Spark User DL on BCC. The actual email message is
> below.
> =============================================================
>
>
> Hi:
>
> I have been working on few example using zeppelin.
>
> I have been trying to find a command that would list all *dataframes/RDDs*
> that has been created in current session. Anyone knows if there is any such
> commands available?
>
> Something similar to SparkSQL to list all temp tables :
> show tables;
>
> Thanks,
> Dhaval
>
>
>
> On Thu, Aug 20, 2015 at 12:49 PM, Dhaval Patel <dh...@gmail.com>
> wrote:
>
>> Hi:
>>
>> I have been working on few example using zeppelin.
>>
>> I have been trying to find a command that would list all
>> *dataframes/RDDs* that has been created in current session. Anyone knows if
>> there is any such commands available?
>>
>> Something similar to SparkSQL to list all temp tables :
>> show tables;
>>
>> Thanks,
>> Dhaval
>>
>
>
Re: How to list all dataframes and RDDs available in current session?
Posted by Dhaval Patel <dh...@gmail.com>.
Apologies!!!!
I accidentally included Spark User DL on BCC. The actual email message is
below.
=============================================================
Hi:
I have been working on few example using zeppelin.
I have been trying to find a command that would list all *dataframes/RDDs*
that has been created in current session. Anyone knows if there is any such
commands available?
Something similar to SparkSQL to list all temp tables :
show tables;
Thanks,
Dhaval
On Thu, Aug 20, 2015 at 12:49 PM, Dhaval Patel <dh...@gmail.com> wrote:
> Hi:
>
> I have been working on few example using zeppelin.
>
> I have been trying to find a command that would list all *dataframes/RDDs*
> that has been created in current session. Anyone knows if there is any such
> commands available?
>
> Something similar to SparkSQL to list all temp tables :
> show tables;
>
> Thanks,
> Dhaval
>