[jira] [Created] (IMPALA-11232) Consider not adding some jars when starting HMS

["Fang-Yu Rao (Jira)" <ji...@apache.org>]

Fang-Yu Rao created IMPALA-11232:
------------------------------------

             Summary: Consider not adding some jars when starting HMS
                 Key: IMPALA-11232
                 URL: https://issues.apache.org/jira/browse/IMPALA-11232
             Project: IMPALA
          Issue Type: Test
          Components: Infrastructure
    Affects Versions: Impala 4.0.0
            Reporter: Fang-Yu Rao
            Assignee: Fang-Yu Rao


Currently when we execute {{run-hive-server.sh}} with the "{{-with_ranger}}" flag, we add all the jar files under the folder of {{$RANGER_HOME/ews/webapp/WEB-INF/lib}} to '{{HADOOP_CLASSPATH}}' before we call the shell script that starts the Hive service, i.e., {{bin/hive}}.

Recall that {{bin/hive}} would attempt to execute '{{hadoop version}}' to determine the version of the underlying Hadoop distribution before starting the Hive Metastore and the HiveServer2. By default, this Hadoop command under the covers would add every path on '{{HADOOP_CLASSPATH}}' to its own environment variable of '{{CLASSPATH}}'. When there are too many paths added to '{{CLASSPATH}}', the Hadoop command would fail with the error message of "{{Argument list too long.}}" and hence the Hive service could not be started.

We execute {{run-hive-server.sh}} with the "{{-with_ranger}}" flag in order to run the end-to-end test of {{test_hive_with_ranger_setup}}. We should consider not adding some (or all) of the jar files to '{{HADOOP_CLASSPATH}}' from the folder described if some (or all) of them are not required to run the end-to-end test so as to avoid the error mentioned above.




--
This message was sent by Atlassian Jira
(v8.20.1#820001)