Access z.show table data in spark

[Paul Brenner <pb...@placeiq.com>]

I often find myself using z.show() to see some portion of a dataframe and then realizing that I want to use some of the values from that table in my next paragraph of code. For example, I may show a table of Lat/Longs and then once I look at it realize that I want to next filter a data frame for the Lat/Long that showed up in the first row.

Is there currently or could there ever be a way to access those values in code? Currently the best I can do is click into the values to copy them and then paste them out. Alternately since I’m using spark I can re-run the query to collect those values out to a variable but that isn’t practical for long running queries.

This is probably a long shot but there is a lot zeppelin is capable of that I don’t know about so figured it was worth asking. Also, something about the way z.show() tables are shown makes it really frustrating to copy and paste (I can’t select values in tables unless I click on them a few times)

Paul Brenner
Head of Data Science
pbrenner@placeiq.com | (217) 390-3033 | www.placeiq.com
twitter @placeiq linkedin /placeiq

Re: Access z.show table data in spark

[Jeff Zhang <zj...@gmail.com>]

Maybe you can use saveAs to save the intermediate result for reuse by other
paragraphs. But it only works for spark.sql for now I believe

[image: image.png]

Paul Brenner <pb...@placeiq.com> 于2021年3月18日周四 下午10:06写道：

> I often find myself using z.show() to see some portion of a dataframe and
> then realizing that I want to use some of the values from that table in my
> next paragraph of code. For example, I may show a table of Lat/Longs and
> then once I look at it realize that I want to next filter a data frame for
> the Lat/Long that showed up in the first row.
>
> Is there currently or could there ever be a way to access those values in
> code? Currently the best I can do is click into the values to copy them and
> then paste them out. Alternately since I’m using spark I can re-run the
> query to collect those values out to a variable but that isn’t practical
> for long running queries.
>
> This is probably a long shot but there is a lot zeppelin is capable of
> that I don’t know about so figured it was worth asking. Also, something
> about the way z.show() tables are shown makes it really frustrating to copy
> and paste (I can’t select values in tables unless I click on them a few
> times)
>
> Paul Brenner
>
> Head of Data Science
> *pbrenner@placeiq.com* <pb...@placeiq.com> | (217) 390-3033 |
> *www.placeiq.com* <https://www.placeiq.com>
> twitter *@placeiq* <https://twitter.com/PlaceIQ> linkedin */placeiq*
> <https://www.linkedin.com/company/placeiq/>
> [image: Check out PlaceIQ's latest COVID-19 research]
> <https://www.placeiq.com/blog/?utm_source=email&utm_medium=signature&utm_campaign=blogsubscriber>
> [image: PlaceIQ: Oracle Data Cloud Premier Data Provider]
> <http://go.placeiq.com/WC01DPG0l0N3Gq000000e00>
>


-- 
Best Regards

Jeff Zhang