You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Dongjoon Hyun (JIRA)" <ji...@apache.org> on 2016/04/30 23:29:12 UTC
[jira] [Updated] (SPARK-15031) Use SparkSession in
Scala/Python/Java example.
[ https://issues.apache.org/jira/browse/SPARK-15031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dongjoon Hyun updated SPARK-15031:
----------------------------------
Priority: Major (was: Trivial)
Description:
This PR aims to update Scala/Python/Java examples by replacing SQLContext with newly added SparkSession. For this, two new `SparkSesion` ctor are added, and also fixes the following examples.
**sql.py**
{code}
- people = sqlContext.jsonFile(path)
+ people = sqlContext.read.json(path)
- people.registerAsTable("people")
+ people.registerTempTable("people")
{code}
**dataframe_example.py**
{code}
- features = df.select("features").map(lambda r: r.features)
+ features = df.select("features").rdd.map(lambda r: r.features)
{code}
Note that the following examples are untouched in this PR since it fails some unknown issue.
- `simple_params_example.py`
- `aft_survival_regression.py`
was:
This PR aims to update Scala/Python examples by replacing SQLContext with newly added SparkSession. Also, this fixes the following examples.
**sql.py**
{code}
- people = sqlContext.jsonFile(path)
+ people = sqlContext.read.json(path)
- people.registerAsTable("people")
+ people.registerTempTable("people")
{code}
**dataframe_example.py**
{code}
- features = df.select("features").map(lambda r: r.features)
+ features = df.select("features").rdd.map(lambda r: r.features)
{code}
Note that the following examples are untouched in this PR since it fails some unknown issue.
- `simple_params_example.py`
- `aft_survival_regression.py`
Summary: Use SparkSession in Scala/Python/Java example. (was: Use SparkSession in Scala/Python example.)
> Use SparkSession in Scala/Python/Java example.
> ----------------------------------------------
>
> Key: SPARK-15031
> URL: https://issues.apache.org/jira/browse/SPARK-15031
> Project: Spark
> Issue Type: Improvement
> Components: Examples
> Reporter: Dongjoon Hyun
>
> This PR aims to update Scala/Python/Java examples by replacing SQLContext with newly added SparkSession. For this, two new `SparkSesion` ctor are added, and also fixes the following examples.
> **sql.py**
> {code}
> - people = sqlContext.jsonFile(path)
> + people = sqlContext.read.json(path)
> - people.registerAsTable("people")
> + people.registerTempTable("people")
> {code}
> **dataframe_example.py**
> {code}
> - features = df.select("features").map(lambda r: r.features)
> + features = df.select("features").rdd.map(lambda r: r.features)
> {code}
> Note that the following examples are untouched in this PR since it fails some unknown issue.
> - `simple_params_example.py`
> - `aft_survival_regression.py`
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org