[jira] [Commented] (HIVE-3431) Resources on non-local file system should be downloaded to temporary directory sometimes

["Phabricator (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13544561#comment-13544561 ] 

Phabricator commented on HIVE-3431:
-----------------------------------

ashutoshc has requested changes to the revision "HIVE-3431 [jira] Resources on non-local file system should be downloaded to temporary directory sometimes".

  Mostly looks good except for unwarranted change of retrieving user name from env instead of current System properties.

  I also realized while reviewing this that we never delete these resources. This may become a problem for HiveServer since in server mode, various user files might get downloaded in this server dir which will never be deleted and thus filling up server disk. I think solution here will be to keep track of these files in SessionState and delete all such files once the query ends. This is pre-existing problem which we need to deal with. Can you file a new jira for this, so that we can follow this up?

INLINE COMMENTS
  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:190 Last line of http://docs.oracle.com/javase/tutorial/essential/environment/env.html
  says :
  To maximize portability, never refer to an environment variable when the same value is available in a system property. For example, if the operating system provides a user name, it will always be available in the system property user.name.

  So, I think change is unwarranted. Please, revert it.

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:943 Per previous comment, this new method than will no longer be needed.
  conf/hive-default.xml.template:1624 Update this also, once you revert previous change.

REVISION DETAIL
  https://reviews.facebook.net/D5199

BRANCH
  DPAL-1872

To: JIRA, ashutoshc, navis
Cc: mgrover

                
> Resources on non-local file system should be downloaded to temporary directory sometimes
> ----------------------------------------------------------------------------------------
>
>                 Key: HIVE-3431
>                 URL: https://issues.apache.org/jira/browse/HIVE-3431
>             Project: Hive
>          Issue Type: Improvement
>          Components: Configuration
>    Affects Versions: 0.10.0
>            Reporter: Navis
>            Assignee: Navis
>            Priority: Trivial
>         Attachments: HIVE-3431.1.patch.txt, HIVE-3431.D5199.2.patch, HIVE-3431.D5199.3.patch
>
>
> "add resource <remote-uri>" command downloads the resource file to location specified by conf "hive.downloaded.resources.dir" in local file system. But when the command above is executed concurrently to hive-server for same file, some client fails by VM crash, which is caused by overwritten file by other requests.
> So there should be a configuration to provide per request location for add resource command, something like "set hiveconf:hive.downloaded.resources.dir=temporary"

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira