You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "Xing Pan (Jira)" <ji...@apache.org> on 2019/12/02 10:14:00 UTC

[jira] [Created] (HUDI-376) AWS Glue dependency issue for EMR 5.28.0

Xing Pan created HUDI-376:
-----------------------------

             Summary: AWS Glue dependency issue for EMR 5.28.0
                 Key: HUDI-376
                 URL: https://issues.apache.org/jira/browse/HUDI-376
             Project: Apache Hudi (incubating)
          Issue Type: Bug
          Components: CLI
            Reporter: Xing Pan


Hi hudi team, it's really encouraging that Hudi is finally officially supported application on AWS EMR. Great job!

I found a *ClassNotFound* exception when using:
{code:java}
/usr/lib/hudi/bin/run_sync_tool.sh
{code}
in emr master.

And I think is due to demand of aws glue data sdk dependency.

So I added a line to run_sync_tool.sh to get a quick fix for this:
{code:java}
HIVE_JARS=$HIVE_JARS:/usr/lib/hive/auxlib/aws-glue-datacatalog-hive2-client.jar:/usr/share/aws/emr/emr-metrics-collector/lib/aws-java-sdk-glue-1.11.475.jar{code}
not sure if any more jars needed, but these two jar fixed my problem.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)