Available Functions in SparkR

["Eskilson,Aleksander" <Al...@Cerner.com>]

I’ve built Spark 1.4.0 for Hadoop 2.6 in a CDH5.4 and am testing SparkR. I’ve loaded up SparkR using the executable in /bin. The library import library(SparkR) seems to no longer import some of the same functions as it did for SparkR before the merge, e.g. textFile, lapply, etc. but it does include sparkR.init, take, and other original functions. How is it planned to access the full set of functions in the repl with the coming version of SparkR?

Thanks,
Alek Eskilson

CONFIDENTIALITY NOTICE This message and any included attachments are from Cerner Corporation and are intended only for the addressee. The information contained in this message is confidential and may constitute inside or non-public information under international, federal, or state securities laws. Unauthorized forwarding, printing, copying, distribution, or use of such information is strictly prohibited and may be unlawful. If you are not the addressee, please promptly delete this message and notify the sender of the delivery error by e-mail or you may call Cerner's corporate offices in Kansas City, Missouri, U.S.A at (+1) (816)221-1024.

Re: Available Functions in SparkR

[Shivaram Venkataraman <sh...@eecs.berkeley.edu>]

For the 1.4 release the DataFrame API will be publicly available and the
documentation at
http://people.apache.org/~pwendell/spark-releases/spark-1.4.0-rc2-docs/sql-programming-guide.html
(Click on the R tab) provides a good summary of the available functions.

As I described in the other email to the dev list, we are still collecting
feedback on a parallel API for SparkR as we feel the RDD API is too
low-level. We would like to hear any use-cases you have as it will be
valuable in designing the API.

Thanks
Shivaram

On Fri, May 22, 2015 at 7:34 AM, Eskilson,Aleksander <
Alek.Eskilson@cerner.com> wrote:

>  I’ve built Spark 1.4.0 for Hadoop 2.6 in a CDH5.4 and am testing SparkR.
> I’ve loaded up SparkR using the executable in /bin. The library import
> library(SparkR) seems to no longer import some of the same functions as it
> did for SparkR before the merge, e.g. textFile, lapply, etc. but it does
> include sparkR.init, take, and other original functions. How is it planned
> to access the full set of functions in the repl with the coming version of
> SparkR?
>
>  Thanks,
> Alek Eskilson
>  CONFIDENTIALITY NOTICE This message and any included attachments are from
> Cerner Corporation and are intended only for the addressee. The information
> contained in this message is confidential and may constitute inside or
> non-public information under international, federal, or state securities
> laws. Unauthorized forwarding, printing, copying, distribution, or use of
> such information is strictly prohibited and may be unlawful. If you are not
> the addressee, please promptly delete this message and notify the sender of
> the delivery error by e-mail or you may call Cerner's corporate offices in
> Kansas City, Missouri, U.S.A at (+1) (816)221-1024.
>