You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@hive.apache.org by "Sahil Takiar (JIRA)" <ji...@apache.org> on 2017/09/15 19:42:00 UTC

[jira] [Created] (HIVE-17546) HoS Explain plan should show if a Work is being cached

Sahil Takiar created HIVE-17546:
-----------------------------------

             Summary: HoS Explain plan should show if a Work is being cached
                 Key: HIVE-17546
                 URL: https://issues.apache.org/jira/browse/HIVE-17546
             Project: Hive
          Issue Type: Improvement
          Components: Physical Optimizer, Spark
            Reporter: Sahil Takiar


Right now there is no way to know if a given {{BaseWork}} is going to be cached by Spark (e.g. {{RDD.cache()}}). The criteria for determined if a Work is being cached is pretty simple, and it should be straight-forward to move the logic into the {{SparkCompiler}}.

This should allow us add an entry in the explain plan to indicate if a {{Work}} is going to be cached or not. This should also allow us to add some explicit q tests just for the RDD cacheing optimization.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)