You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Eli Super <el...@gmail.com> on 2016/01/25 11:00:08 UTC

SparkSQL : "select non null values from column"

Hi

I try to select all values but not NULL values from column contains NULL
values

with

sqlContext.sql("select my_column from my_table where my_column <> null
").show(15)

or

sqlContext.sql("select my_column from my_table where my_column != null
").show(15)

I get empty result


Thanks !

Re: SparkSQL : "select non null values from column"

Posted by Deng Ching-Mallete <oc...@apache.org>.

Hi,

Have you tried using IS NOT NULL for the where condition?

Thanks,
Deng

On Mon, Jan 25, 2016 at 7:00 PM, Eli Super <el...@gmail.com> wrote:

> Hi
>
> I try to select all values but not NULL values from column contains NULL
> values
>
> with
>
> sqlContext.sql("select my_column from my_table where my_column <> null
> ").show(15)
>
> or
>
> sqlContext.sql("select my_column from my_table where my_column != null
> ").show(15)
>
> I get empty result
>
>
> Thanks !
>