You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@spark.apache.org by JaeSung Jun <ja...@gmail.com> on 2015/04/14 16:13:19 UTC

DDL parser class parsing DDL in spark-sql cli

Hi,

Wile I've been walking through spark-sql source code, I typed the following
HiveQL:

CREATE EXTERNAL TABLE user (uid STRING, age INT, gender STRING, job STRING,
ts STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' LOCATION
'/hive/user';

, and I finally came across ddl.scala after analysing a couple of scala
files.

What I found here is createTable method, which doesn't parse the above
statement.
Please let me know which class is responsible for parsing the above
statement.

Thanks
Jason

Re: DDL parser class parsing DDL in spark-sql cli

Posted by JaeSung Jun <ja...@gmail.com>.

Thanks Michael,

I was wondering how HiveContext.sql() is hooked up HiveQL..I'll have a look
at it.
much appreciated.

Thanks
Jason

On 15 April 2015 at 04:15, Michael Armbrust <mi...@databricks.com> wrote:

> HiveQL
> <https://github.com/apache/spark/blob/master/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveQl.scala>
>
> On Tue, Apr 14, 2015 at 7:13 AM, JaeSung Jun <ja...@gmail.com> wrote:
>
>> Hi,
>>
>> Wile I've been walking through spark-sql source code, I typed the
>> following
>> HiveQL:
>>
>> CREATE EXTERNAL TABLE user (uid STRING, age INT, gender STRING, job
>> STRING,
>> ts STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' LOCATION
>> '/hive/user';
>>
>> , and I finally came across ddl.scala after analysing a couple of scala
>> files.
>>
>> What I found here is createTable method, which doesn't parse the above
>> statement.
>> Please let me know which class is responsible for parsing the above
>> statement.
>>
>> Thanks
>> Jason
>>
>
>

Re: DDL parser class parsing DDL in spark-sql cli

Posted by Michael Armbrust <mi...@databricks.com>.

HiveQL
<https://github.com/apache/spark/blob/master/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveQl.scala>

On Tue, Apr 14, 2015 at 7:13 AM, JaeSung Jun <ja...@gmail.com> wrote:

> Hi,
>
> Wile I've been walking through spark-sql source code, I typed the following
> HiveQL:
>
> CREATE EXTERNAL TABLE user (uid STRING, age INT, gender STRING, job STRING,
> ts STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' LOCATION
> '/hive/user';
>
> , and I finally came across ddl.scala after analysing a couple of scala
> files.
>
> What I found here is createTable method, which doesn't parse the above
> statement.
> Please let me know which class is responsible for parsing the above
> statement.
>
> Thanks
> Jason
>