You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Jerry Wong <je...@gmail.com> on 2016/06/07 17:18:49 UTC

Spark SQL: org.apache.spark.sql.AnalysisException: cannot resolve "some columns" given input columns.

Hi,

Two JSON files but one of them miss some columns, like

{"firstName": "Jack", "lastName": "Nelson"}
{"firstName": "Landy", "middleName": "Ken", "lastName": "Yong"}

slqContext.sql("select firstName as first_name, middleName as middle_name,
lastName as last_name from jsonTable)

But there are an error
org.apache.spark.sql.AnalysisException: cannot resolve 'middleName' given
input columns firstName, lastName;

Can anybody give me your wisdom or any suggestions?

Thanks!
Jerry

Re: Spark SQL: org.apache.spark.sql.AnalysisException: cannot resolve "some columns" given input columns.

Posted by Ted Yu <yu...@gmail.com>.

Please see:

[SPARK-13953][SQL] Specifying the field name for corrupted record via
option at JSON datasource

FYI

On Tue, Jun 7, 2016 at 10:18 AM, Jerry Wong <je...@gmail.com>
wrote:

> Hi,
>
> Two JSON files but one of them miss some columns, like
>
> {"firstName": "Jack", "lastName": "Nelson"}
> {"firstName": "Landy", "middleName": "Ken", "lastName": "Yong"}
>
> slqContext.sql("select firstName as first_name, middleName as middle_name,
> lastName as last_name from jsonTable)
>
> But there are an error
> org.apache.spark.sql.AnalysisException: cannot resolve 'middleName' given
> input columns firstName, lastName;
>
> Can anybody give me your wisdom or any suggestions?
>
> Thanks!
> Jerry
>
>