[jira] [Resolved] (SPARK-12255) why "cache table a as select * from b" will do shuffle,and create 2 stages

["Sean Owen (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-12255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sean Owen resolved SPARK-12255.
-------------------------------
    Resolution: Invalid

Questions go to user@spark.apache.org not JIRA

> why "cache table a as select * from b" will do shuffle,and create 2 stages
> --------------------------------------------------------------------------
>
>                 Key: SPARK-12255
>                 URL: https://issues.apache.org/jira/browse/SPARK-12255
>             Project: Spark
>          Issue Type: Question
>         Environment: spark-1.4.1-bin-hadoop2.4
>            Reporter: ant_nebula
>         Attachments: pic1.jpg, pic2.jpg
>
>
> why "cache table a as select * from b" will do shuffle,and create 2 stages.
> example:
> table "ods_pay_consume" is from "KafkaUtils.createDirectStream"
> {code}
> hiveContext.sql("cache table dwd_pay_consume as select * from ods_pay_consume")
> {code}
> this code will create DAG as pic1.jsp
> {code}
> hiveContext.sql(""cache table dw_game_server_recharge as select * from dwd_pay_consume")
> {code}
> this code will create DAG as pic2.jsp,and this similar caculate from the beginning,"cache table dwd_pay_consume" is not effect.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org