[jira] [Created] (SPARK-22189) Number of jobs created while querying partitioned table in hive using spark

["Astha Arya (JIRA)" <ji...@apache.org>]

Astha Arya created SPARK-22189:
----------------------------------

             Summary: Number of jobs created while querying partitioned table in hive using spark
                 Key: SPARK-22189
                 URL: https://issues.apache.org/jira/browse/SPARK-22189
             Project: Spark
          Issue Type: Question
          Components: SQL
    Affects Versions: 1.6.0
            Reporter: Astha Arya


I am using Spark SQL 
Spark version - 1.6.0
Hive 1.1.0-cdh5.9.0
When I run hiveContext.sql, creates 2 another job for my case i.e. 3 jobs in total for querying hive for a partitioned table. Whereas when i run the same query on hive using spark as execution engine, it makes only one job. 
Also, the driver logs show that it lists all the partitions which most likely shouldnt happen because it slows down my execution. 
Is this a bug? Is there any way to reduce the number of jobs and also not list all the partitions each time I query the same table ? 




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org