You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Ryan Skraba (Jira)" <ji...@apache.org> on 2019/11/04 13:45:00 UTC

[jira] [Created] (AVRO-2616) Do not use Hadoop FS for local files with avro-tools

Ryan Skraba created AVRO-2616:
---------------------------------

             Summary: Do not use Hadoop FS for local files with avro-tools
                 Key: AVRO-2616
                 URL: https://issues.apache.org/jira/browse/AVRO-2616
             Project: Apache Avro
          Issue Type: Bug
          Components: java
            Reporter: Ryan Skraba


The avro-tools jar includes the Hadoop dependencies inside the fat jar.  This is probably so that the CLI can operate with files that are located on the cluster (or other URIs that have Hadoop FileSystem implementations).

This is useful if the user is accessing HDFS with the same version as the Hadoop FileSystem, and *mostly* neutral if the user is accessing other URIs, including the local filesystem.

Hadoop doesn't currently officially support any version [after JDK 8|https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+Java+Versions] (at time of writing).  We might want to access the local filesystem bypassing the Hadoop jars to avoid requiring this dependency when it is not needed. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)