You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by 霍战锋 <hu...@gmail.com> on 2019/08/13 12:00:14 UTC
Sreate temp table or view by using location
Hi,
I'm trying to use spark SQL to define a temp table which can be
destroyed automatically with the session. But when I using the SQL as
below, I can't query any valid row, meanwhile, it works when I delete the
word 'temp'. Is there anyone can tell me how to write the right SQL?
It doesn't work like this one.
spark.sql("create temp view people (name string, age int) using csv
options(sep=',',inferSchema='true',ignoreLeadingWhiteSpace='true')
location 'src/main/resources/people.txt'")
It works like this one, but it's not a temp table.
spark.sql("create view people (name string, age int) using csv
options(sep=',',inferSchema='true',ignoreLeadingWhiteSpace='true')
location 'src/main/resources/people.txt'")
As a repetition, I would like to use SQL to get the same result as
'dataset.createOrReplaceTempView(table
name)'.
Thank you.
Best Regards
Re: Sreate temp table or view by using location
Posted by 霍战锋 <hu...@gmail.com>.
It works by using "create temp view" and "options(path='something')"
together, thanks.
spark.sql("""create temp view people (name string, age int) using csv
options(sep=',',inferSchema='true',ignoreLeadingWhiteSpace='true',path='src/main/resources/people.txt')""")
Best Regards
霍战锋 <hu...@gmail.com> 于2019年8月13日周二 下午8:03写道：
> Sorry for the typo. The title is 'Create temp table or view by using
> location'.
> Best Regards
>
>
> 霍战锋 <hu...@gmail.com> 于2019年8月13日周二 下午8:00写道：
>
>> Hi,
>>
>> I'm trying to use spark SQL to define a temp table which can be
>> destroyed automatically with the session. But when I using the SQL as
>> below, I can't query any valid row, meanwhile, it works when I delete the
>> word 'temp'. Is there anyone can tell me how to write the right SQL?
>>
>> It doesn't work like this one.
>> spark.sql("create temp view people (name string, age int) using csv options(sep=',',inferSchema='true',ignoreLeadingWhiteSpace='true') location 'src/main/resources/people.txt'")
>>
>> It works like this one, but it's not a temp table.
>> spark.sql("create view people (name string, age int) using csv options(sep=',',inferSchema='true',ignoreLeadingWhiteSpace='true') location 'src/main/resources/people.txt'")
>>
>> As a repetition, I would like to use SQL to get the same result as
>> 'dataset.createOrReplaceTempView(table name)'.
>>
>> Thank you.
>>
>> Best Regards
>>
>
Re: Sreate temp table or view by using location
Posted by 霍战锋 <hu...@gmail.com>.
Sorry for the typo. The title is 'Create temp table or view by using
location'.
Best Regards
霍战锋 <hu...@gmail.com> 于2019年8月13日周二 下午8:00写道：
> Hi,
>
> I'm trying to use spark SQL to define a temp table which can be
> destroyed automatically with the session. But when I using the SQL as
> below, I can't query any valid row, meanwhile, it works when I delete the
> word 'temp'. Is there anyone can tell me how to write the right SQL?
>
> It doesn't work like this one.
> spark.sql("create temp view people (name string, age int) using csv options(sep=',',inferSchema='true',ignoreLeadingWhiteSpace='true') location 'src/main/resources/people.txt'")
>
> It works like this one, but it's not a temp table.
> spark.sql("create view people (name string, age int) using csv options(sep=',',inferSchema='true',ignoreLeadingWhiteSpace='true') location 'src/main/resources/people.txt'")
>
> As a repetition, I would like to use SQL to get the same result as
> 'dataset.createOrReplaceTempView(table name)'.
>
> Thank you.
>
> Best Regards
>