You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Justin Pihony <ju...@gmail.com> on 2015/03/04 21:53:48 UTC
Spark SQL Static Analysis
I am pretty sure that I saw a presentation where SparkSQL could be executed
with static analysis, however I cannot find the presentation now, nor can I
find any documentation or research papers on the topic. So, I am curious if
there is indeed any work going on for this topic. The two things I would be
interested in would be to be able to gain compile time safety, as well as
gain the ability to work on my data as a type instead of a row (ie,
result.map(x=>x.Age) instead of having to use Row.get)
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-SQL-Static-Analysis-tp21918.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: Spark SQL Static Analysis
Posted by Justin Pihony <ju...@gmail.com>.
Thanks!
On Wed, Mar 4, 2015 at 3:58 PM, Michael Armbrust <mi...@databricks.com>
wrote:
> It is somewhat out of data, but here is what we have so far:
> https://github.com/marmbrus/sql-typed
>
> On Wed, Mar 4, 2015 at 12:53 PM, Justin Pihony <ju...@gmail.com>
> wrote:
>
>> I am pretty sure that I saw a presentation where SparkSQL could be
>> executed
>> with static analysis, however I cannot find the presentation now, nor can
>> I
>> find any documentation or research papers on the topic. So, I am curious
>> if
>> there is indeed any work going on for this topic. The two things I would
>> be
>> interested in would be to be able to gain compile time safety, as well as
>> gain the ability to work on my data as a type instead of a row (ie,
>> result.map(x=>x.Age) instead of having to use Row.get)
>>
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-spark-user-list.1001560.n3.nabble.com/Spark-SQL-Static-Analysis-tp21918.html
>> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>> For additional commands, e-mail: user-help@spark.apache.org
>>
>>
>
Re: Spark SQL Static Analysis
Posted by Michael Armbrust <mi...@databricks.com>.
It is somewhat out of data, but here is what we have so far:
https://github.com/marmbrus/sql-typed
On Wed, Mar 4, 2015 at 12:53 PM, Justin Pihony <ju...@gmail.com>
wrote:
> I am pretty sure that I saw a presentation where SparkSQL could be executed
> with static analysis, however I cannot find the presentation now, nor can I
> find any documentation or research papers on the topic. So, I am curious if
> there is indeed any work going on for this topic. The two things I would be
> interested in would be to be able to gain compile time safety, as well as
> gain the ability to work on my data as a type instead of a row (ie,
> result.map(x=>x.Age) instead of having to use Row.get)
>
>
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Spark-SQL-Static-Analysis-tp21918.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>