[jira] [Updated] (AVRO-983) maven-avro-plugin: Allow maven schema compiler to support external reference in an another avro schema file

["Tom White (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AVRO-983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tom White updated AVRO-983:
---------------------------

    Attachment: AVRO-983.patch

Here's a simpler patch that follows Doug's suggestion of reusing Schema.Parser. I haven't written a unit test but I did verify it manually by writing a schema that referenced another schema and checked that without the patch the plugin failed and with the patch it successfully compiled all the files.

There is however a big caveat: cross references only work if the type is defined in a file that is compiled before it is referenced.

So if a.avsc defines type A which depends on type B defined in b.avsc then compilation will fail. That's because when a.avsc is compiled first (since it is lexically before b.avsc) B will not have been read.

It's possible to rename avsc files to make this work, so this patch is an improvement; but it would be better to improve this in the future to either

* give more control over the order in which the files are compiled, like the command line version. At the moment they are specified as directories in the Maven POM, so the order is the lexical ordering of the files in the directory.
* or enhance the compiler to support type references between files. This is more involved and would require a two pass approach.
                
> maven-avro-plugin: Allow maven schema compiler to support external reference in an another avro schema file
> -----------------------------------------------------------------------------------------------------------
>
>                 Key: AVRO-983
>                 URL: https://issues.apache.org/jira/browse/AVRO-983
>             Project: Avro
>          Issue Type: Improvement
>          Components: java
>    Affects Versions: 1.6.1
>            Reporter: Cédric Torcq
>         Attachments: AVRO-983.patch, SchemaMojo.java.patch
>
>   Original Estimate: 0.25h
>  Remaining Estimate: 0.25h
>
> Avro Maven plugin doesn't support an external reference in an another Avro schema file even this schema is available in the same src directory
> Example :
> {code:javascript} 
> [
> 	{
> 		"namespace": "com.afklm.karma.demand.inter",
> 		"name": "DatedFlightLegInterType",
> 		"type": "record",
> 		"fields":
> 			[
> 				{ "name": "legOrgArp", "type": "string" },
> 				{ "name": "legDestArp", "type": "string" },
> 				{ "name": "legDepDate", "type": "long" },
> 				{ "name": "flightNb", "type": "string" },
> 				{ "name": "dfl", "type": "com.afklm.karma.demand.output.DatedFlightLegType" }
> 			]
> 	},
> 	{
> 		"namespace": "com.afklm.karma.demand.inter",
> 		"name": "AirportAirportDatedFlightLegInterType",
> 		"type": "record",
> 		"fields":
> 			[
> 				{ "name": "aa", "type": "AirportAirportInterType" },
> 				{ "name": "legOrgArp", "type": "string" },
> 				{ "name": "legDestArp", "type": "string" },
> 				{ "name": "legDepDate", "type": "long" },
> 				{ "name": "flightNb", "type": "string" }
> 			]
> 	}
> ]
> {code} 
> generate a fatal error :
> {code} 
> org.apache.avro.SchemaParseException: "com.afklm.karma.demand.output.DatedFlightLegType" is not a defined name. The type of t
> he "dfl" field must be a defined name or a {"type": ...} expression.
>         at org.apache.avro.Schema.parse(Schema.java:1131)
>         at org.apache.avro.Schema.parse(Schema.java:1200)
>         at org.apache.avro.Schema$Parser.parse(Schema.java:968)
>         at org.apache.avro.Schema$Parser.parse(Schema.java:945)
>         at org.apache.avro.mojo.SchemaMojo.doCompile(SchemaMojo.java:58)
>         at org.apache.avro.mojo.AbstractAvroMojo.compileFiles(AbstractAvroMojo.java:137)
>         at org.apache.avro.mojo.AbstractAvroMojo.execute(AbstractAvroMojo.java:107)
> {code}
> I propose a patch to support also this case :
> In org.apache.avro.mojo.SchemaMojo (line 55):
> 		// File src = new File(sourceDirectory, filename);
> 		// Schema.Parser parser = new Schema.Parser();
> 		// Schema schema = parser.parse(src);
> 		// SpecificCompiler compiler = new SpecificCompiler(schema);
> 		// compiler.setTemplateDir(templateDirectory);
> 		// compiler.setStringType(StringType.valueOf(stringType));
> 		// compiler.compileSchema(src,
> 		// outputDirectory);
> 		
> 		
>         DirectoryScanner scanner = new DirectoryScanner();
>         scanner.setBasedir(sourceDirectory);
>         scanner.setIncludes( getIncludes() );
>         scanner.addDefaultExcludes();
>         scanner.scan();
>         File[] avroFiles = new File[scanner.getIncludedFiles().length];
>         for (int i = 0; i < avroFiles.length; i++) {
>         	avroFiles[i] = new File( scanner.getBasedir(), scanner.getIncludedFiles()[i] );
> 		}
>         SpecificCompiler.compileSchema(avroFiles, outputDirectory);

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira