You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "jin xing (JIRA)" <ji...@apache.org> on 2018/03/13 14:41:00 UTC

[jira] [Created] (SPARK-23669) Executors fetch jars and name the jars with md5 prefix

jin xing created SPARK-23669:
--------------------------------

             Summary: Executors fetch jars and name the jars with md5 prefix
                 Key: SPARK-23669
                 URL: https://issues.apache.org/jira/browse/SPARK-23669
             Project: Spark
          Issue Type: New Feature
          Components: Spark Core
    Affects Versions: 2.3.0
         Environment: In our cluster, there are lots of UDF jars, some of them have the same filename but different path, for example:

```

hdfs://A/B/udf.jar  -> udfA

hdfs://C/D/udf.jar -> udfB

```

When user uses udfA and udfB in same sql, executor will fetch both \{hdfs://A/B/udf.jar} and \{hdfs://C/D/udf.jar}. There will be a conflict for the same name. 

 

Can we config to fetch jars and save with a filename with MD5 prefix, so there will be no conflict.
            Reporter: jin xing






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org