You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by unk1102 <um...@gmail.com> on 2015/09/10 16:39:48 UTC
How to enable Tungsten in Spark 1.5 for Spark SQL?
Hi Spark 1.5 looks promising how do we enable project tungsten for spark sql
or is it enabled by default please guide.
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-enable-Tungsten-in-Spark-1-5-for-Spark-SQL-tp24642.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: How to enable Tungsten in Spark 1.5 for Spark SQL?
Posted by Umesh Kacha <um...@gmail.com>.
Nice Ted thanks much highest performance without any configuration changes
amazed! Looking forward to running Spark 1.5 on my 2 tb skewed data which
involves group by union etc any other tips if you know for spark 1.5
On Sep 10, 2015 8:12 PM, "Ted Yu" <yu...@gmail.com> wrote:
> Please see the following
> in sql/core/src/main/scala/org/apache/spark/sql/SQLConf.scala :
>
> val TUNGSTEN_ENABLED = booleanConf("spark.sql.tungsten.enabled",
> defaultValue = Some(true),
> doc = "When true, use the optimized Tungsten physical execution
> backend which explicitly " +
> "manages memory and dynamically generates bytecode for
> expression evaluation.")
>
> val CODEGEN_ENABLED = booleanConf("spark.sql.codegen",
> defaultValue = Some(true), // use TUNGSTEN_ENABLED as default
> doc = "When true, code will be dynamically generated at runtime for
> expression evaluation in" +
> " a specific query.",
> isPublic = false)
>
> val UNSAFE_ENABLED = booleanConf("spark.sql.unsafe.enabled",
> defaultValue = Some(true), // use TUNGSTEN_ENABLED as default
> doc = "When true, use the new optimized Tungsten physical execution
> backend.",
> isPublic = false)
>
> Cheers
>
> On Thu, Sep 10, 2015 at 7:39 AM, unk1102 <um...@gmail.com> wrote:
>
>> Hi Spark 1.5 looks promising how do we enable project tungsten for spark
>> sql
>> or is it enabled by default please guide.
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-spark-user-list.1001560.n3.nabble.com/How-to-enable-Tungsten-in-Spark-1-5-for-Spark-SQL-tp24642.html
>> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>> For additional commands, e-mail: user-help@spark.apache.org
>>
>>
>
Re: How to enable Tungsten in Spark 1.5 for Spark SQL?
Posted by Ted Yu <yu...@gmail.com>.
Please see the following
in sql/core/src/main/scala/org/apache/spark/sql/SQLConf.scala :
val TUNGSTEN_ENABLED = booleanConf("spark.sql.tungsten.enabled",
defaultValue = Some(true),
doc = "When true, use the optimized Tungsten physical execution backend
which explicitly " +
"manages memory and dynamically generates bytecode for expression
evaluation.")
val CODEGEN_ENABLED = booleanConf("spark.sql.codegen",
defaultValue = Some(true), // use TUNGSTEN_ENABLED as default
doc = "When true, code will be dynamically generated at runtime for
expression evaluation in" +
" a specific query.",
isPublic = false)
val UNSAFE_ENABLED = booleanConf("spark.sql.unsafe.enabled",
defaultValue = Some(true), // use TUNGSTEN_ENABLED as default
doc = "When true, use the new optimized Tungsten physical execution
backend.",
isPublic = false)
Cheers
On Thu, Sep 10, 2015 at 7:39 AM, unk1102 <um...@gmail.com> wrote:
> Hi Spark 1.5 looks promising how do we enable project tungsten for spark
> sql
> or is it enabled by default please guide.
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/How-to-enable-Tungsten-in-Spark-1-5-for-Spark-SQL-tp24642.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>