[jira] [Comment Edited] (PARQUET-2142) parquet-cli without hadoop throws java.lang.NoSuchMethodError on any parquet file access command

["Timothy Miller (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PARQUET-2142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17527556#comment-17527556 ] 

Timothy Miller edited comment on PARQUET-2142 at 4/26/22 1:19 PM:
------------------------------------------------------------------

I added -verbose:class to the java command line, and I found this in the output:
{noformat}
[Loaded org.apache.parquet.avro.AvroSchemaConverter from file:/local/home/theosib/packages/PARQUET-2140/parquet-mr/parquet-cli/target/parquet-cli-1.13.0-SNAPSHOT-runtime.jar] {noformat}
This isn't correct. It should be loading AvroSchemaConverter from target/dependency/parquet-avro-1.13.0-SNAPSHOT.jar. Evidently the instructions are wrong. I changed the run command to this:
{noformat}
java -cp 'target/parquet-cli-1.13.0-SNAPSHOT.jar:target/dependency/*' org.apache.parquet.cli.Main cat ~/guid.parquet{noformat}
And the NoSuchMethodError went away. Now I get the Unknown error and java.lang.RuntimeException that we see in PARQUET-2143.

 


was (Author: JIRAUSER287471):
I added -verbose:class to the java command line, and I found this in the output:
{noformat}
[Loaded org.apache.parquet.avro.AvroSchemaConverter from file:/local/home/theosib/packages/PARQUET-2140/parquet-mr/parquet-cli/target/parquet-cli-1.13.0-SNAPSHOT-runtime.jar] {noformat}
This isn't correct. It should be loading AvroSchemaConverter from target/dependency/parquet-avro-1.13.0-SNAPSHOT.jar. Evidently the instructions are wrong. I changed the run command to this:
{noformat}
java -cp 'target/parquet-cli-1.13.0-SNAPSHOT.jar:target/dependency/*:target/dependency/parquet-avro-1.13.0-SNAPSHOT.jar' org.apache.parquet.cli.Main cat ~/guid.parquet{noformat}
And the NoSuchMethodError went away. Now I get the Unknown error and java.lang.RuntimeException that we see in PARQUET-2143.

 

> parquet-cli without hadoop throws java.lang.NoSuchMethodError on any parquet file access command
> ------------------------------------------------------------------------------------------------
>
>                 Key: PARQUET-2142
>                 URL: https://issues.apache.org/jira/browse/PARQUET-2142
>             Project: Parquet
>          Issue Type: Bug
>          Components: parquet-cli
>    Affects Versions: 1.13.0
>         Environment: Amazon Linux 2, Java 8
>            Reporter: Timothy Miller
>            Priority: Blocker
>
> I can't do even basic things with parquet-cli from 1.13.0-SNAPSHOT.
> Steps to reproduce:
> {noformat}
> git clone <parquet-mr repo> 
> cd parquet-mr
> mvn clean install -DskipTests
> cd parquet-cli
> mvn clean install -DskipTests
> mvn dependency:copy-dependencies
> java -cp 'target/*:target/dependency/*' org.apache.parquet.cli.Main cat <parquet file>{noformat}
> Results:
> {noformat}
> Exception in thread "main" java.lang.NoSuchMethodError: org.apache.parquet.avro.AvroSchemaConverter.convert(Lorg/apache/parquet/schema/MessageType;)Lorg/apache/avro/Schema;
>     at org.apache.parquet.cli.util.Schemas.fromParquet(Schemas.java:89)
>     at org.apache.parquet.cli.BaseCommand.getAvroSchema(BaseCommand.java:405)
>     at org.apache.parquet.cli.commands.CatCommand.run(CatCommand.java:66)
>     at org.apache.parquet.cli.Main.run(Main.java:157)
>     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
>     at org.apache.parquet.cli.Main.main(Main.java:187){noformat}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)