You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Stephen Boesch <ja...@gmail.com> on 2017/08/10 20:46:26 UTC
SQL specific documentation for recent Spark releases
While the DataFrame/DataSets are useful in many circumstances they are
cumbersome for many types of complex sql queries.
Is there an up to date *SQL* reference - i.e. not DataFrame DSL operations
- for version 2.2?
An example of what is not clear: what constructs are supported within
select count( predicate) from some_table
when using spark sql.
But in general the reference guide and programming guide for SQL seems to
be difficult to locate - seemingly in favor of the DataFrame/DataSets.
Re: SQL specific documentation for recent Spark releases
Posted by Reynold Xin <rx...@databricks.com>.
This PR should help you in the next release:
https://github.com/apache/spark/pull/18702
On Thu, Aug 10, 2017 at 7:46 PM, Stephen Boesch <ja...@gmail.com> wrote:
>
> The correct link is https://docs.databricks.com/
> spark/latest/spark-sql/index.html .
>
> This link does have the core syntax such as the BNF for the DDL and DML
> and SELECT. It does *not *have a reference for date / string / numeric
> functions: is there any such reference at this point? It is not sufficient
> to peruse the DSL list of functions since the usage is different (and
> sometimes the names as well) than from the DSL.
>
> thanks
> stephenb
>
> 2017-08-10 14:49 GMT-07:00 Jules Damji <dm...@comcast.net>:
>
>> I refer to docs.databricks.com/Spark/latest/Spark-sql/index.html.
>>
>> Cheers
>> Jules
>>
>> Sent from my iPhone
>> Pardon the dumb thumb typos :)
>>
>> > On Aug 10, 2017, at 1:46 PM, Stephen Boesch <ja...@gmail.com> wrote:
>> >
>> >
>> > While the DataFrame/DataSets are useful in many circumstances they are
>> cumbersome for many types of complex sql queries.
>> >
>> > Is there an up to date *SQL* reference - i.e. not DataFrame DSL
>> operations - for version 2.2?
>> >
>> > An example of what is not clear: what constructs are supported within
>> >
>> > select count( predicate) from some_table
>> >
>> > when using spark sql.
>> >
>> > But in general the reference guide and programming guide for SQL seems
>> to be difficult to locate - seemingly in favor of the DataFrame/DataSets.
>>
>>
>
Re: SQL specific documentation for recent Spark releases
Posted by Stephen Boesch <ja...@gmail.com>.
The correct link is
https://docs.databricks.com/spark/latest/spark-sql/index.html .
This link does have the core syntax such as the BNF for the DDL and DML and
SELECT. It does *not *have a reference for date / string / numeric
functions: is there any such reference at this point? It is not sufficient
to peruse the DSL list of functions since the usage is different (and
sometimes the names as well) than from the DSL.
thanks
stephenb
2017-08-10 14:49 GMT-07:00 Jules Damji <dm...@comcast.net>:
> I refer to docs.databricks.com/Spark/latest/Spark-sql/index.html.
>
> Cheers
> Jules
>
> Sent from my iPhone
> Pardon the dumb thumb typos :)
>
> > On Aug 10, 2017, at 1:46 PM, Stephen Boesch <ja...@gmail.com> wrote:
> >
> >
> > While the DataFrame/DataSets are useful in many circumstances they are
> cumbersome for many types of complex sql queries.
> >
> > Is there an up to date *SQL* reference - i.e. not DataFrame DSL
> operations - for version 2.2?
> >
> > An example of what is not clear: what constructs are supported within
> >
> > select count( predicate) from some_table
> >
> > when using spark sql.
> >
> > But in general the reference guide and programming guide for SQL seems
> to be difficult to locate - seemingly in favor of the DataFrame/DataSets.
>
>
Re: SQL specific documentation for recent Spark releases
Posted by Jules Damji <dm...@comcast.net>.
I refer to docs.databricks.com/Spark/latest/Spark-sql/index.html.
Cheers
Jules
Sent from my iPhone
Pardon the dumb thumb typos :)
> On Aug 10, 2017, at 1:46 PM, Stephen Boesch <ja...@gmail.com> wrote:
>
>
> While the DataFrame/DataSets are useful in many circumstances they are cumbersome for many types of complex sql queries.
>
> Is there an up to date *SQL* reference - i.e. not DataFrame DSL operations - for version 2.2?
>
> An example of what is not clear: what constructs are supported within
>
> select count( predicate) from some_table
>
> when using spark sql.
>
> But in general the reference guide and programming guide for SQL seems to be difficult to locate - seemingly in favor of the DataFrame/DataSets.
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org