You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@avro.apache.org by "fanguad (Jira)" <ji...@apache.org> on 2023/01/13 13:30:00 UTC
[jira] [Updated] (AVRO-3706) AVDL nested imports cannot be resolved if path contains spaces
[ https://issues.apache.org/jira/browse/AVRO-3706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
fanguad updated AVRO-3706:
--------------------------
Description:
There is an issue when running the IDL tool if the IDL files have nested import statements, and the path includes a space (or probably any character that must be URL encoded). After the first level of import, the files can no longer be found. Consider the following three files in a {{avro bug}} directory:
{{root.avdl}}
{code:java}
protocol Root {
import idl "level1.avdl";
} {code}
{{level1.avdl}}
{code:java}
protocol Level1 {
import idl "level2.avdl";
} {code}
{{level2.avdl}}
{code:java}
protocol Level2 {
} {code}
executing {{java -jar avro-tools-1.10.2.jar idl "avro bug/root.avdl"}} results in the expected
{code:java}
{
"protocol" : "Root",
"namespace" : null,
"types" : [ ],
"messages" : { }
} {code}
whereas {{java -jar avro-tools-1.11.1.jar idl "avro bug/root.avdl"}} causes an exception (the bug is also present in 1.11.0 with a slightly different stack trace)
{code:java}
Exception in thread "main" org.apache.avro.compiler.idl.ParseException: Error importing level2.avdl: java.io.FileNotFoundException: level2.avdl, at line 2, column 26
at org.apache.avro.compiler.idl.Idl.error(Idl.java:88)
at org.apache.avro.compiler.idl.Idl.ImportIdl(Idl.java:537)
at org.apache.avro.compiler.idl.Idl.ProtocolBody(Idl.java:411)
at org.apache.avro.compiler.idl.Idl.ProtocolDeclaration(Idl.java:286)
at org.apache.avro.compiler.idl.Idl.CompilationUnit(Idl.java:167)
at org.apache.avro.compiler.idl.Idl.ImportIdl(Idl.java:535)
at org.apache.avro.compiler.idl.Idl.ProtocolBody(Idl.java:411)
at org.apache.avro.compiler.idl.Idl.ProtocolDeclaration(Idl.java:286)
at org.apache.avro.compiler.idl.Idl.CompilationUnit(Idl.java:167)
at org.apache.avro.tool.IdlTool.run(IdlTool.java:61)
at org.apache.avro.tool.Main.run(Main.java:67)
at org.apache.avro.tool.Main.main(Main.java:56)
{code}
The same issue occurs if the files are located in "avro" but the current folder has spaces (ie, the absolute path to the avdl files contains spaces).
The bug appears to have been introduced in this commit [https://github.com/apache/avro/commit/3fe5e306eaa43bdc06cef291321a8c04058a6be9#diff-376865a18691674de38817b4eea7e64c2c3848094a4730030c67f57f54f1028a] in {{idl.jj}} lines 130 and 169. The second time that line 130 is reached, it has an absolute path that is already URL-encoded. In 1.10.2, the code {{new File(input.getPath()).getParentFile()}} returns a String like {{[file:///home/user/avro%20bug/]}} whereas in 1.11.0 and later {{new File(input.getPath()).getParentFile().toURI()}} returns a String like {{[file:///home/user/avro%2520bug]}} - note that URL encoding has happened twice resulting in %2520 and there is no trailing slash. When resolved on line 169, this results in {{[file:///home/user/level2.avdl]}} instead of the correct {{{}[file:///home/user/avro%20bug/level2.avdl]{}}}.
This bug can be easily worked around by changing the location of the avdl files to one without spaces or other characters that must be URL encoded.
was:
There is an issue when running the IDL tool if the IDL files have nested import statements, and the path includes a space (or probably any character that must be URL encoded). After the first level of import, the files can no longer be found. Consider the following three files in a {{avro bug}} directory:
{{root.avdl}}
{code:java}
protocol Root {
import idl "level1.avdl";
} {code}
{{level1.avdl}}
{code:java}
protocol Level1 {
import idl "level2.avdl";
} {code}
{{level2.avdl}}
{code:java}
protocol Level2 {
} {code}
executing {{java -jar avro-tools-1.10.2.jar idl "avro bug/root.avdl"}} results in the expected
{code:java}
{
"protocol" : "Root",
"namespace" : null,
"types" : [ ],
"messages" : { }
} {code}
whereas {{java -jar avro-tools-1.11.1.jar idl "avro bug/root.avdl"}} causes an exception (the bug is also present in 1.11.0 with a slightly different stack trace)
{code:java}
Exception in thread "main" org.apache.avro.compiler.idl.ParseException: Error importing level2.avdl: java.io.FileNotFoundException: level2.avdl, at line 2, column 26
at org.apache.avro.compiler.idl.Idl.error(Idl.java:88)
at org.apache.avro.compiler.idl.Idl.ImportIdl(Idl.java:537)
at org.apache.avro.compiler.idl.Idl.ProtocolBody(Idl.java:411)
at org.apache.avro.compiler.idl.Idl.ProtocolDeclaration(Idl.java:286)
at org.apache.avro.compiler.idl.Idl.CompilationUnit(Idl.java:167)
at org.apache.avro.compiler.idl.Idl.ImportIdl(Idl.java:535)
at org.apache.avro.compiler.idl.Idl.ProtocolBody(Idl.java:411)
at org.apache.avro.compiler.idl.Idl.ProtocolDeclaration(Idl.java:286)
at org.apache.avro.compiler.idl.Idl.CompilationUnit(Idl.java:167)
at org.apache.avro.tool.IdlTool.run(IdlTool.java:61)
at org.apache.avro.tool.Main.run(Main.java:67)
at org.apache.avro.tool.Main.main(Main.java:56)
{code}
The same issue occurs if the files are located in "avro" but the current folder has spaces (ie, the absolute path to the avdl files contains spaces).
The bug appears to have been introduced in this commit [https://github.com/apache/avro/commit/3fe5e306eaa43bdc06cef291321a8c04058a6be9#diff-376865a18691674de38817b4eea7e64c2c3848094a4730030c67f57f54f1028a] in {{idl.jj}} lines 130 and 169. The second time that line 130 is reached, it has an absolute path that is already URL-encoded. In 1.10.2, the code {{new File(input.getPath()).getParentFile()}} returns a String like {{[file:///home/user/avro%20bug/]}} whereas in 1.11.0 and later {{new File(input.getPath()).getParentFile().toURI()}} returns a String like {{[file:///home/user/avro%2520bug]}} - note that URL encoding has happened twice resulting in %2520 and there is no trailing slash. When resolved on line 169, this results in {{[file:///home/user/level2.avdl]}} instead of the correct {{{}[file:///home/user/avro%20bug/level2.avdl]{}}}.
This bug can be easily worked around by changing the location of the avdl files to one without spaces or other characters that must be URL encoded.
> AVDL nested imports cannot be resolved if path contains spaces
> --------------------------------------------------------------
>
> Key: AVRO-3706
> URL: https://issues.apache.org/jira/browse/AVRO-3706
> Project: Apache Avro
> Issue Type: Bug
> Components: java
> Affects Versions: 1.11.0, 1.11.1
> Reporter: fanguad
> Priority: Minor
>
> There is an issue when running the IDL tool if the IDL files have nested import statements, and the path includes a space (or probably any character that must be URL encoded). After the first level of import, the files can no longer be found. Consider the following three files in a {{avro bug}} directory:
> {{root.avdl}}
> {code:java}
> protocol Root {
> import idl "level1.avdl";
> } {code}
> {{level1.avdl}}
> {code:java}
> protocol Level1 {
> import idl "level2.avdl";
> } {code}
> {{level2.avdl}}
> {code:java}
> protocol Level2 {
> } {code}
> executing {{java -jar avro-tools-1.10.2.jar idl "avro bug/root.avdl"}} results in the expected
>
> {code:java}
> {
> "protocol" : "Root",
> "namespace" : null,
> "types" : [ ],
> "messages" : { }
> } {code}
> whereas {{java -jar avro-tools-1.11.1.jar idl "avro bug/root.avdl"}} causes an exception (the bug is also present in 1.11.0 with a slightly different stack trace)
>
> {code:java}
> Exception in thread "main" org.apache.avro.compiler.idl.ParseException: Error importing level2.avdl: java.io.FileNotFoundException: level2.avdl, at line 2, column 26
> at org.apache.avro.compiler.idl.Idl.error(Idl.java:88)
> at org.apache.avro.compiler.idl.Idl.ImportIdl(Idl.java:537)
> at org.apache.avro.compiler.idl.Idl.ProtocolBody(Idl.java:411)
> at org.apache.avro.compiler.idl.Idl.ProtocolDeclaration(Idl.java:286)
> at org.apache.avro.compiler.idl.Idl.CompilationUnit(Idl.java:167)
> at org.apache.avro.compiler.idl.Idl.ImportIdl(Idl.java:535)
> at org.apache.avro.compiler.idl.Idl.ProtocolBody(Idl.java:411)
> at org.apache.avro.compiler.idl.Idl.ProtocolDeclaration(Idl.java:286)
> at org.apache.avro.compiler.idl.Idl.CompilationUnit(Idl.java:167)
> at org.apache.avro.tool.IdlTool.run(IdlTool.java:61)
> at org.apache.avro.tool.Main.run(Main.java:67)
> at org.apache.avro.tool.Main.main(Main.java:56)
> {code}
>
> The same issue occurs if the files are located in "avro" but the current folder has spaces (ie, the absolute path to the avdl files contains spaces).
>
> The bug appears to have been introduced in this commit [https://github.com/apache/avro/commit/3fe5e306eaa43bdc06cef291321a8c04058a6be9#diff-376865a18691674de38817b4eea7e64c2c3848094a4730030c67f57f54f1028a] in {{idl.jj}} lines 130 and 169. The second time that line 130 is reached, it has an absolute path that is already URL-encoded. In 1.10.2, the code {{new File(input.getPath()).getParentFile()}} returns a String like {{[file:///home/user/avro%20bug/]}} whereas in 1.11.0 and later {{new File(input.getPath()).getParentFile().toURI()}} returns a String like {{[file:///home/user/avro%2520bug]}} - note that URL encoding has happened twice resulting in %2520 and there is no trailing slash. When resolved on line 169, this results in {{[file:///home/user/level2.avdl]}} instead of the correct {{{}[file:///home/user/avro%20bug/level2.avdl]{}}}.
>
> This bug can be easily worked around by changing the location of the avdl files to one without spaces or other characters that must be URL encoded.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)