[jira] [Created] (SPARK-5318) Add ability to control partition count in SparkSql

["Idan Zalzberg (JIRA)" <ji...@apache.org>]

Idan Zalzberg created SPARK-5318:
------------------------------------

             Summary: Add ability to control partition count in SparkSql
                 Key: SPARK-5318
                 URL: https://issues.apache.org/jira/browse/SPARK-5318
             Project: Spark
          Issue Type: New Feature
          Components: SQL
            Reporter: Idan Zalzberg


When using SparkSql, e.g. sqlContext.sql("..."), spark might need to read hadoop files.
However, unlike the hadoopFile API, there is no documented way to set the minimal partition count when reading.
There is an undocumented way, though, using "mapred.map.tasks" in hiveConf

I suggest we make a documented way to do it, in the exact same way (possibly with a better name)




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org