You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by xingye <xi...@hotmail.com> on 2016/09/09 17:29:22 UTC
Assign values to existing column in SparkR
I have some questions about assign values to a spark dataframe. I want to assign values to an existing column of a spark dataframe but if I assign the value directly, I got the following error.df$c_mon<-0Error: class(value) == "Column" || is.null(value) is not TRUEIs there a way to solve this?
Re: Assign values to existing column in SparkR
Posted by Felix Cheung <fe...@hotmail.com>.
If you are to set a column to 0 (essentially remove and replace the existing one) you would need to put a column on the right hand side:
> df <- as.DataFrame(iris)
> head(df)
Sepal_Length Sepal_Width Petal_Length Petal_Width Species
1 5.1 3.5 1.4 0.2 setosa
2 4.9 3.0 1.4 0.2 setosa
3 4.7 3.2 1.3 0.2 setosa
4 4.6 3.1 1.5 0.2 setosa
5 5.0 3.6 1.4 0.2 setosa
6 5.4 3.9 1.7 0.4 setosa
> df$Petal_Length <- 0
Error: class(value) == "Column" || is.null(value) is not TRUE
> df$Petal_Length <- lit(0)
> head(df)
Sepal_Length Sepal_Width Petal_Length Petal_Width Species
1 5.1 3.5 0 0.2 setosa
2 4.9 3.0 0 0.2 setosa
3 4.7 3.2 0 0.2 setosa
4 4.6 3.1 0 0.2 setosa
5 5.0 3.6 0 0.2 setosa
6 5.4 3.9 0 0.4 setosa
_____________________________
From: Deepak Sharma <de...@gmail.com>>
Sent: Friday, September 9, 2016 12:29 PM
Subject: Re: Assign values to existing column in SparkR
To: xingye <xi...@hotmail.com>>
Cc: <us...@spark.apache.org>>
Data frames are immutable in nature , so i don't think you can directly assign or change values on the column.
Thanks
Deepak
On Fri, Sep 9, 2016 at 10:59 PM, xingye <xi...@hotmail.com>> wrote:
I have some questions about assign values to a spark dataframe. I want to assign values to an existing column of a spark dataframe but if I assign the value directly, I got the following error.
1. df$c_mon<-0
2. Error: class(value) == "Column" || is.null(value) is not TRUE
Is there a way to solve this?
--
Thanks
Deepak
www.bigdatabig.com<http://www.bigdatabig.com>
www.keosha.net<http://www.keosha.net>
Re: Assign values to existing column in SparkR
Posted by Deepak Sharma <de...@gmail.com>.
Data frames are immutable in nature , so i don't think you can directly
assign or change values on the column.
Thanks
Deepak
On Fri, Sep 9, 2016 at 10:59 PM, xingye <xi...@hotmail.com> wrote:
> I have some questions about assign values to a spark dataframe. I want to
> assign values to an existing column of a spark dataframe but if I assign
> the value directly, I got the following error.
>
>
> 1. df$c_mon<-0
> 2. Error: class(value) == "Column" || is.null(value) is not TRUE
>
> Is there a way to solve this?
>
--
Thanks
Deepak
www.bigdatabig.com
www.keosha.net