You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/11/23 19:42:52 UTC
[GitHub] [arrow] asheeshgarg opened a new issue, #14721: FlatBuffer Issue
asheeshgarg opened a new issue, #14721:
URL: https://github.com/apache/arrow/issues/14721
### Describe the bug, including details regarding any error messages, version, and platform.
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>log4j-over-slf4j</artifactId>
</dependency>
<dependency>
<groupId>org.apache.iceberg</groupId>
<artifactId>iceberg-common</artifactId>
<version>1.0.0</version>
</dependency>
<dependency>
<groupId>org.apache.iceberg</groupId>
<artifactId>iceberg-core</artifactId>
<version>1.0.0</version>
</dependency>
<dependency>
<groupId>org.apache.iceberg</groupId>
<artifactId>iceberg-data</artifactId>
<version>1.0.0</version>
</dependency>
<dependency>
<groupId>org.apache.iceberg</groupId>
<artifactId>iceberg-arrow</artifactId>
<version>1.0.0</version>
</dependency>
<dependency>
<groupId>org.apache.iceberg</groupId>
<artifactId>iceberg-hive-runtime</artifactId>
<version>1.0.0</version>
</dependency>
<dependency>
<groupId>org.apache.iceberg</groupId>
<artifactId>iceberg-hive-metastore</artifactId>
<version>1.0.0</version>
</dependency>
<dependency>
<groupId>org.apache.hive</groupId>
<artifactId>hive-metastore</artifactId>
<version>3.1.3</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<version>3.3.1</version>
</dependency>
<dependency>
<groupId>org.apache.hive</groupId>
<artifactId>hive-common</artifactId>
<version>2.3.9</version>
</dependency>
<dependency>
<groupId>org.apache.hive</groupId>
<artifactId>hive-storage-api</artifactId>
<version>2.8.1</version>
</dependency>
<dependency>
<groupId>org.apache.thrift</groupId>
<artifactId>libthrift</artifactId>
<version>0.13.0</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-mapreduce-client-core</artifactId>
<version>3.3.4</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-aws</artifactId>
<version>3.3.1</version>
</dependency>
<dependency>
<groupId>com.google.flatbuffers</groupId>
<artifactId>flatbuffers-java</artifactId>
<version>1.12.0</version>
</dependency>
<dependency>
<groupId>org.apache.arrow</groupId>
<artifactId>arrow-vector</artifactId>
<version>9.0.0</version>
</dependency>
<dependency>
<groupId>org.apache.arrow</groupId>
<artifactId>arrow-memory-netty</artifactId>
<version>9.0.0</version>
</dependency>
<dependency>
<groupId>org.apache.arrow</groupId>
<artifactId>flight-core</artifactId>
<version>9.0.0</version>
</dependency>
<dependency>
<groupId>org.apache.arrow</groupId>
<artifactId>flight-sql</artifactId>
<version>9.0.0</version>
</dependency>
<dependency>
<groupId>org.apache.arrow</groupId>
<artifactId>flight-grpc</artifactId>
<version>9.0.0</version>
</dependency>
<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
<version>3.21.8</version>
</dependency>
<dependency>
<groupId>commons-cli</groupId>
<artifactId>commons-cli</artifactId>
<version>1.4</version>
</dependency>
java.lang.NoSuchMethodError: com.google.flatbuffers.FlatBufferBuilder.createString(Ljava/lang/CharSequence;)I
at org.apache.arrow.vector.types.pojo.Field.getField (Field.java:189)
at org.apache.arrow.vector.types.pojo.Schema.getSchema (Schema.java:208)
at org.apache.arrow.vector.ipc.message.MessageSerializer.serializeMetadata (MessageSerializer.java:187)
at org.apache.arrow.flight.ArrowMessage.<init> (ArrowMessage.java:152)
at org.apache.arrow.flight.DictionaryUtils.generateSchemaMessages (DictionaryUtils.java:64)
at org.apache.arrow.flight.OutboundStreamListenerImpl.start (OutboundStreamListenerImpl.java:61)
at org.apache.arrow.flight.OutboundStreamListener.start (OutboundStreamListener.java:67)
at org.apache.arrow.flight.FlightClient.startPut (FlightClient.java:242)
at org.apache.arrow.flight.FlightClient.startPut (FlightClient.java:225)
at com.icebergread.CookbookProducer.main (CookbookProducer.java:190)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:254)
at java.lang.Thread.run (Thread.java:834)
### Component
Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lwhite1 commented on issue #14721: FlatBuffer Issue
Posted by GitBox <gi...@apache.org>.
lwhite1 commented on issue #14721:
URL: https://github.com/apache/arrow/issues/14721#issuecomment-1329479175
Also, please take a look at https://stackoverflow.com/questions/69236212/java-lang-nosuchmethoderror-com-google-flatbuffers-flatbufferbuilder-createstri
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] asheeshgarg closed issue #14721: FlatBuffer Issue
Posted by GitBox <gi...@apache.org>.
asheeshgarg closed issue #14721: FlatBuffer Issue
URL: https://github.com/apache/arrow/issues/14721
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lwhite1 commented on issue #14721: FlatBuffer Issue
Posted by GitBox <gi...@apache.org>.
lwhite1 commented on issue #14721:
URL: https://github.com/apache/arrow/issues/14721#issuecomment-1329554423
Arrow vector uses the flat buffer version 1.12.0 (granted, I'm looking at arrow version 11-SNAPSHOT), as defined in the arrow root pom
<dep.fbs.version>1.12.0</dep.fbs.version>
Do you have another dependency somewhere on flatbuffers that is pointing to a different flatbuffer version?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] asheeshgarg commented on issue #14721: FlatBuffer Issue
Posted by GitBox <gi...@apache.org>.
asheeshgarg commented on issue #14721:
URL: https://github.com/apache/arrow/issues/14721#issuecomment-1325575846
While using the Example
try(VectorSchemaRoot vectorSchemaRoot = VectorSchemaRoot.create(schema, allocator);
VarCharVector varCharVector = (VarCharVector) vectorSchemaRoot.getVector("name")) {
varCharVector.allocateNew(3);
varCharVector.setSafe(0,"Ronald".getBytes("utf-8"));
varCharVector.setSafe(1, "David".getBytes("utf-8"));
varCharVector.setSafe(2, "Francisco".getBytes("utf-8"));
vectorSchemaRoot.setRowCount(3);
FlightClient.ClientStreamListener listener = flightClient.startPut(
FlightDescriptor.path("profiles"),
vectorSchemaRoot, new AsyncPutListener());
listener.putNext();
varCharVector.setSafe(0, "Manuel".getBytes());
varCharVector.setSafe(1, "Felipe".getBytes());
varCharVector.setSafe(2, "JJ".getBytes());
vectorSchemaRoot.setRowCount(3);
listener.putNext();
listener.completed();
listener.getResult();
System.out.println("C2: Client (Populate Data): Wrote 2 batches with 3 rows each");
}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] asheeshgarg commented on issue #14721: FlatBuffer Issue
Posted by GitBox <gi...@apache.org>.
asheeshgarg commented on issue #14721:
URL: https://github.com/apache/arrow/issues/14721#issuecomment-1325632803
Its causing at the time of put
FlightClient.ClientStreamListener listener = flightClient.startPut(
FlightDescriptor.path("profiles"),
vectorSchemaRoot, new AsyncPutListener());
Any pointers to resolve this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lwhite1 commented on issue #14721: FlatBuffer Issue
Posted by GitBox <gi...@apache.org>.
lwhite1 commented on issue #14721:
URL: https://github.com/apache/arrow/issues/14721#issuecomment-1329518587
see the answer here for details, but you might try adding the needed flat buffer dependency directly to your project pom.
https://stackoverflow.com/questions/24962607/multiple-versions-of-the-same-dependency-in-maven
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] asheeshgarg commented on issue #14721: FlatBuffer Issue
Posted by GitBox <gi...@apache.org>.
asheeshgarg commented on issue #14721:
URL: https://github.com/apache/arrow/issues/14721#issuecomment-1334038447
@lwhite1 its working for me know.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lwhite1 commented on issue #14721: FlatBuffer Issue
Posted by GitBox <gi...@apache.org>.
lwhite1 commented on issue #14721:
URL: https://github.com/apache/arrow/issues/14721#issuecomment-1329583579
FWIW, I see the reference in the arrow-format pom, which has no version defined, but that will just pick up the version defined in the parent pom (arrow-root) which uses the same vision as arrow-vector. I think we're looking for another project with a dependency on flatbuffers that is different from 1.12.0. (There are several newer and older versions in maven so it wouldn't be too surprising if there was a conflict.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] asheeshgarg commented on issue #14721: FlatBuffer Issue
Posted by GitBox <gi...@apache.org>.
asheeshgarg commented on issue #14721:
URL: https://github.com/apache/arrow/issues/14721#issuecomment-1329533530
@lwhite1
<dependency>
<groupId>org.apache.arrow</groupId>
<artifactId>arrow-vector</artifactId>
<version>9.0.0</version>
</dependency>
Figure out this jar is causing the issue
Arrow vector also has reference to flatbuffer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] asheeshgarg commented on issue #14721: FlatBuffer Issue
Posted by GitBox <gi...@apache.org>.
asheeshgarg commented on issue #14721:
URL: https://github.com/apache/arrow/issues/14721#issuecomment-1329326675
Any update on this I tried upgrading to 10.0.0 jars as well but still hitting the same issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] assignUser commented on issue #14721: FlatBuffer Issue
Posted by GitBox <gi...@apache.org>.
assignUser commented on issue #14721:
URL: https://github.com/apache/arrow/issues/14721#issuecomment-1326484941
Could you please use code blocks to format your logs etc. so they are more readable. Could you please also list your java version.
I am not versed in java but the [method does exists](https://github.com/google/flatbuffers/blob/6df40a2471737b27271bdd9b900ab5f3aec746c7/java/com/google/flatbuffers/FlatBufferBuilder.java#L537) in 1.12.0 which you seem to have installed.
cc @lwhite1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lwhite1 commented on issue #14721: FlatBuffer Issue
Posted by GitBox <gi...@apache.org>.
lwhite1 commented on issue #14721:
URL: https://github.com/apache/arrow/issues/14721#issuecomment-1329465645
@asheeshgarg Sorry. Thurs/Friday was a major holiday in the US. I'm not familiar with the Flight code but I will take a look
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] asheeshgarg commented on issue #14721: FlatBuffer Issue
Posted by GitBox <gi...@apache.org>.
asheeshgarg commented on issue #14721:
URL: https://github.com/apache/arrow/issues/14721#issuecomment-1327685695
Java version is 17.0.4
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] asheeshgarg commented on issue #14721: FlatBuffer Issue
Posted by GitBox <gi...@apache.org>.
asheeshgarg commented on issue #14721:
URL: https://github.com/apache/arrow/issues/14721#issuecomment-1329499100
@lwhite1 thanks for the pointer I removed all the dependency and just run simple flight example it worked. Now need to figure out which Jar is adding the different flat buffer version. Any way to find it easily or force to use the version of the flatbuffer needed by flight?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org