[jira] [Created] (HIVE-4986) Problem in creating configration in org.apache.hadoop.hive.serde2.avro.AvroSerdeUtils

["Subroto Sanyal (JIRA)" <ji...@apache.org>]

Subroto Sanyal created HIVE-4986:
------------------------------------

             Summary: Problem in creating configration in org.apache.hadoop.hive.serde2.avro.AvroSerdeUtils
                 Key: HIVE-4986
                 URL: https://issues.apache.org/jira/browse/HIVE-4986
             Project: Hive
          Issue Type: Bug
            Reporter: Subroto Sanyal


While using AvroSerDe but, I am facing the problem:
{noformat}
java.lang.IllegalArgumentException: Wrong FS: hdfs://xxx:8020/abc/bbc/test.avsc, expected: file:///
	at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:381)
	at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:55)
	at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:393)
	at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
	at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:125)
	at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
	at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:427)
	at org.apache.hadoop.hive.serde2.avro.AvroSerdeUtils.getSchemaFromHDFS(AvroSerdeUtils.java:105)
	at org.apache.hadoop.hive.serde2.avro.AvroSerdeUtils.determineSchemaOrThrowException(AvroSerdeUtils.java:70)
	at org.apache.hadoop.hive.serde2.avro.AvroSerdeUtils.determineSchemaOrReturnErrorSchema(AvroSerdeUtils.java:87)
	at org.apache.hadoop.hive.serde2.avro.AvroSerDe.initialize(AvroSerDe.java:59)
        at test.subroto.HiveUtil.getDeserializer(HiveUtil.java:102)
{noformat}

While debugging the problem I saw a piece of code in:
{code:title=org.apache.hadoop.hive.serde2.avro.AvroSerdeUtils.java|borderStyle=solid}

public static Schema More ...determineSchemaOrThrowException(Properties properties)
           throws IOException, AvroSerdeException {
     String schemaString = properties.getProperty(SCHEMA_LITERAL);
     if(schemaString != null && !schemaString.equals(SCHEMA_NONE))
       return Schema.parse(schemaString);
 
     // Try pulling directly from URL
     schemaString = properties.getProperty(SCHEMA_URL);
     if(schemaString == null || schemaString.equals(SCHEMA_NONE))
       throw new AvroSerdeException(EXCEPTION_MESSAGE);
 
     try {
       if(schemaString.toLowerCase().startsWith("hdfs://"))
         return getSchemaFromHDFS(schemaString, new Configuration());
     } catch(IOException ioe) {
       throw new AvroSerdeException("Unable to read schema from HDFS: " + schemaString, ioe);
     }
 
     return Schema.parse(new URL(schemaString).openStream());
   }
{code} 

Here the configuration passed from the Deserilaizer is completely ignored and a new Configuration object is created. 
In my opinion the AvroSerDe should honor the Configuration being passed to Deserializer interface ; rather than creating a new one.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira