You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "hnwyllmm (via GitHub)" <gi...@apache.org> on 2025/05/20 14:24:33 UTC
[PR] fix get length of byte[] in TryCopyLastError [arrow-java]
hnwyllmm opened a new pull request, #760:
URL: https://github.com/apache/arrow-java/pull/760
## What's Changed
Please fill in a description of the changes here.
We should get the length of byte[] by `GetArrayLength` but not `strlen` which may cause invalid memory access.
Closes #759 .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-759: Get length of byte[] in TryCopyLastError [arrow-java]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on PR #760:
URL: https://github.com/apache/arrow-java/pull/760#issuecomment-3021351564
Ok. I'll merge this, but the CI really needs to be fixed...I just don't have time for arrow-java these days :/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-759: Get length of byte[] in TryCopyLastError [arrow-java]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on PR #760:
URL: https://github.com/apache/arrow-java/pull/760#issuecomment-2994637005
ah right, CI is broken...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-759: Get length of byte[] in TryCopyLastError [arrow-java]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on PR #760:
URL: https://github.com/apache/arrow-java/pull/760#issuecomment-3017641465
There's still more:
```
Error: Failed to execute goal com.diffplug.spotless:spotless-maven-plugin:2.44.4:check (spotless-check) on project arrow-c-data: The following files had format violations:
Error: src/test/java/org/apache/arrow/c/ExceptionTest.java
Error: @@ -51,7 +51,7 @@
Error: ····final·List<Object>·batches·=·new·ArrayList<>();
Error:
Error: ····try·(BufferAllocator·allocator·=·new·RootAllocator();
Error: -·········VectorSchemaRoot·root·=·VectorSchemaRoot.create(schema,·allocator))·{
Error: +········VectorSchemaRoot·root·=·VectorSchemaRoot.create(schema,·allocator))·{
Error:
Error: ······final·String·exceptionMessage·=·"This·is·a·message·for·testing·exception.";
Error:
Error: @@ -66,7 +66,7 @@
Error: ······ArrowReader·source·=·new·ExceptionMemoryArrowReader(allocator,·schema,·batches);
Error:
Error: ······try·(final·ArrowArrayStream·stream·=·ArrowArrayStream.allocateNew(allocator);
Error: -···········final·VectorSchemaRoot·importRoot·=·VectorSchemaRoot.create(schema,·allocator))·{
Error: +··········final·VectorSchemaRoot·importRoot·=·VectorSchemaRoot.create(schema,·allocator))·{
Error: ········final·VectorLoader·loader·=·new·VectorLoader(importRoot);
Error: ········Data.exportArrayStream(allocator,·source,·stream);
Error:
Error: @@ -85,10 +85,7 @@
Error: ····private·final·DictionaryProvider·provider;
Error: ····private·int·nextBatch;
Error:
Error: -····ExceptionMemoryArrowReader(
Error: -········BufferAllocator·allocator,
Error: -········Schema·schema,
Error: -········List<Object>·batches)·{
Error: +····ExceptionMemoryArrowReader(BufferAllocator·allocator,·Schema·schema,·List<Object>·batches)·{
Error: ······super(allocator);
Error: ······this.schema·=·schema;
Error: ······this.batches·=·batches;
Error: Run 'mvn spotless:apply' to fix these violations.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-759: Get length of byte[] in TryCopyLastError [arrow-java]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on code in PR #760:
URL: https://github.com/apache/arrow-java/pull/760#discussion_r2119873853
##########
c/src/test/java/org/apache/arrow/c/ExceptionTest.java:
##########
@@ -0,0 +1,172 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.arrow.c;
+
+import static org.assertj.core.api.Assertions.assertThat;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+import org.apache.arrow.memory.BufferAllocator;
+import org.apache.arrow.memory.RootAllocator;
+import org.apache.arrow.vector.IntVector;
+import org.apache.arrow.vector.VectorLoader;
+import org.apache.arrow.vector.VectorSchemaRoot;
+import org.apache.arrow.vector.VectorUnloader;
+import org.apache.arrow.vector.dictionary.Dictionary;
+import org.apache.arrow.vector.dictionary.DictionaryProvider;
+import org.apache.arrow.vector.ipc.ArrowReader;
+import org.apache.arrow.vector.ipc.message.ArrowRecordBatch;
+import org.apache.arrow.vector.types.pojo.ArrowType;
+import org.apache.arrow.vector.types.pojo.Field;
+import org.apache.arrow.vector.types.pojo.Schema;
+import org.junit.jupiter.api.Test;
+
+final class ExceptionTest {
+ @Test
+ public void testException() throws IOException {
+ final Schema schema =
+ new Schema(Collections.singletonList(Field.nullable("ints", new ArrowType.Int(32, true))));
+ final List<Object> batches = new ArrayList<>();
+
+ try (BufferAllocator allocator = new RootAllocator();
+ VectorSchemaRoot root = VectorSchemaRoot.create(schema, allocator)) {
+ final IntVector ints = (IntVector) root.getVector(0);
+ VectorUnloader unloader = new VectorUnloader(root);
+
+ root.allocateNew();
+ ints.setSafe(0, 1);
+ ints.setSafe(1, 2);
+ ints.setSafe(2, 4);
+ ints.setSafe(3, 8);
+ root.setRowCount(4);
+ batches.add(unloader.getRecordBatch());
+
+ final String exceptionMessage = "java.lang.RuntimeException: Error occurred while getting next schema root.\n\tat org.apache.arrow.adapter.jdbc.ArrowVectorIterator.next(ArrowVectorIterator.java:205)\n\tat com.oceanbase.external.jdbc.JdbcScanner.loadNextBatch(JdbcScanner.java:73)\n\tat org.apache.arrow.c.ArrayStreamExporter$ExportedArrayStreamPrivateData.getNext(ArrayStreamExporter.java:72)\nCaused by: java.lang.RuntimeException: Error occurred while consuming data.\n\tat org.apache.arrow.adapter.jdbc.ArrowVectorIterator.consumeData(ArrowVectorIterator.java:127)\n\tat org.apache.arrow.adapter.jdbc.ArrowVectorIterator.load(ArrowVectorIterator.java:178)\n\tat org.apache.arrow.adapter.jdbc.ArrowVectorIterator.next(ArrowVectorIterator.java:198)\n\t... 2 more\nCaused by: java.lang.OutOfMemoryError: Java heap space\n";
Review Comment:
Um, this is a little excessive. Can we just use a short canary string?
##########
c/src/test/java/org/apache/arrow/c/ExceptionTest.java:
##########
@@ -0,0 +1,172 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.arrow.c;
+
+import static org.assertj.core.api.Assertions.assertThat;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+import org.apache.arrow.memory.BufferAllocator;
+import org.apache.arrow.memory.RootAllocator;
+import org.apache.arrow.vector.IntVector;
+import org.apache.arrow.vector.VectorLoader;
+import org.apache.arrow.vector.VectorSchemaRoot;
+import org.apache.arrow.vector.VectorUnloader;
+import org.apache.arrow.vector.dictionary.Dictionary;
+import org.apache.arrow.vector.dictionary.DictionaryProvider;
+import org.apache.arrow.vector.ipc.ArrowReader;
+import org.apache.arrow.vector.ipc.message.ArrowRecordBatch;
+import org.apache.arrow.vector.types.pojo.ArrowType;
+import org.apache.arrow.vector.types.pojo.Field;
+import org.apache.arrow.vector.types.pojo.Schema;
+import org.junit.jupiter.api.Test;
+
+final class ExceptionTest {
+ @Test
+ public void testException() throws IOException {
+ final Schema schema =
+ new Schema(Collections.singletonList(Field.nullable("ints", new ArrowType.Int(32, true))));
+ final List<Object> batches = new ArrayList<>();
+
+ try (BufferAllocator allocator = new RootAllocator();
+ VectorSchemaRoot root = VectorSchemaRoot.create(schema, allocator)) {
+ final IntVector ints = (IntVector) root.getVector(0);
+ VectorUnloader unloader = new VectorUnloader(root);
+
+ root.allocateNew();
+ ints.setSafe(0, 1);
+ ints.setSafe(1, 2);
+ ints.setSafe(2, 4);
+ ints.setSafe(3, 8);
+ root.setRowCount(4);
+ batches.add(unloader.getRecordBatch());
+
+ final String exceptionMessage = "java.lang.RuntimeException: Error occurred while getting next schema root.\n\tat org.apache.arrow.adapter.jdbc.ArrowVectorIterator.next(ArrowVectorIterator.java:205)\n\tat com.oceanbase.external.jdbc.JdbcScanner.loadNextBatch(JdbcScanner.java:73)\n\tat org.apache.arrow.c.ArrayStreamExporter$ExportedArrayStreamPrivateData.getNext(ArrayStreamExporter.java:72)\nCaused by: java.lang.RuntimeException: Error occurred while consuming data.\n\tat org.apache.arrow.adapter.jdbc.ArrowVectorIterator.consumeData(ArrowVectorIterator.java:127)\n\tat org.apache.arrow.adapter.jdbc.ArrowVectorIterator.load(ArrowVectorIterator.java:178)\n\tat org.apache.arrow.adapter.jdbc.ArrowVectorIterator.next(ArrowVectorIterator.java:198)\n\t... 2 more\nCaused by: java.lang.OutOfMemoryError: Java heap space\n";
+
+ RuntimeException ex = new RuntimeException(exceptionMessage);
+ batches.add(ex);
+
+ ArrowReader source = new ExceptionMemoryArrowReader(allocator, schema, batches);
+
+ try (final ArrowArrayStream stream = ArrowArrayStream.allocateNew(allocator);
+ final VectorSchemaRoot importRoot = VectorSchemaRoot.create(schema, allocator)) {
+ final VectorLoader loader = new VectorLoader(importRoot);
+ Data.exportArrayStream(allocator, source, stream);
+
+ try (final ArrowReader reader = Data.importArrayStream(allocator, stream)) {
+ assertThat(reader.getVectorSchemaRoot().getSchema()).isEqualTo(schema);
+
+ for (Object batch : batches) {
+ try {
+ reader.loadNextBatch();
+ } catch (Exception e) {
+ Throwable exToCheck = e;
+ while (exToCheck != null && !(exToCheck instanceof RuntimeException)) {
+ exToCheck = exToCheck.getCause();
+ }
+ if (exToCheck != null) {
+ assertThat(exToCheck.getMessage()).isEqualTo(exceptionMessage);
+ }
+ continue;
+ }
+ loader.load((ArrowRecordBatch) batch);
+
+ assertThat(reader.getVectorSchemaRoot().getRowCount()).isEqualTo(root.getRowCount());
+ }
+ }
+ }
+ }
Review Comment:
We need to assert that we did indeed get an exception.
##########
c/src/test/java/org/apache/arrow/c/ExceptionTest.java:
##########
@@ -0,0 +1,172 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.arrow.c;
+
+import static org.assertj.core.api.Assertions.assertThat;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+import org.apache.arrow.memory.BufferAllocator;
+import org.apache.arrow.memory.RootAllocator;
+import org.apache.arrow.vector.IntVector;
+import org.apache.arrow.vector.VectorLoader;
+import org.apache.arrow.vector.VectorSchemaRoot;
+import org.apache.arrow.vector.VectorUnloader;
+import org.apache.arrow.vector.dictionary.Dictionary;
+import org.apache.arrow.vector.dictionary.DictionaryProvider;
+import org.apache.arrow.vector.ipc.ArrowReader;
+import org.apache.arrow.vector.ipc.message.ArrowRecordBatch;
+import org.apache.arrow.vector.types.pojo.ArrowType;
+import org.apache.arrow.vector.types.pojo.Field;
+import org.apache.arrow.vector.types.pojo.Schema;
+import org.junit.jupiter.api.Test;
+
+final class ExceptionTest {
+ @Test
+ public void testException() throws IOException {
Review Comment:
Please indicate somewhere that this is a regression test for #759.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix get length of byte[] in TryCopyLastError [arrow-java]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on PR #760:
URL: https://github.com/apache/arrow-java/pull/760#issuecomment-2896248990
> The error message in CI doesn't seem to be related to my PR:
>
> ```
> /opt/rh/devtoolset-10/root/usr/libexec/gcc/x86_64-redhat-linux/10/ld: cannot find -lutf8proc
> collect2: error: ld returned 1 exit status
> ```
>
> linkage: https://github.com/apache/arrow-java/actions/runs/15140171347/job/42594091462?pr=760
>
> Do you have some suggestions?
Could you open a separated issue for it?
Let's fix it as a separated task.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-759: Get length of byte[] in TryCopyLastError [arrow-java]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on code in PR #760:
URL: https://github.com/apache/arrow-java/pull/760#discussion_r2133853714
##########
c/src/test/java/org/apache/arrow/c/ExceptionTest.java:
##########
@@ -0,0 +1,184 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.arrow.c;
+
+import static org.assertj.core.api.Assertions.assertThat;
+
+import java.io.IOException;
+import java.io.PrintWriter;
+import java.io.StringWriter;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.Objects;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+import org.apache.arrow.c.jni.CDataJniException;
+import org.apache.arrow.memory.BufferAllocator;
+import org.apache.arrow.memory.RootAllocator;
+import org.apache.arrow.vector.IntVector;
+import org.apache.arrow.vector.VectorLoader;
+import org.apache.arrow.vector.VectorSchemaRoot;
+import org.apache.arrow.vector.VectorUnloader;
+import org.apache.arrow.vector.dictionary.Dictionary;
+import org.apache.arrow.vector.dictionary.DictionaryProvider;
+import org.apache.arrow.vector.ipc.ArrowReader;
+import org.apache.arrow.vector.ipc.message.ArrowRecordBatch;
+import org.apache.arrow.vector.types.pojo.ArrowType;
+import org.apache.arrow.vector.types.pojo.Field;
+import org.apache.arrow.vector.types.pojo.Schema;
+import org.junit.jupiter.api.Test;
+
+// Regression test for https://github.com/apache/arrow-java/issues/759
+final class ExceptionTest {
+ @Test
+ public void testException() throws IOException {
+ final Schema schema =
+ new Schema(Collections.singletonList(Field.nullable("ints", new ArrowType.Int(32, true))));
+ final List<Object> batches = new ArrayList<>();
+
+ try (BufferAllocator allocator = new RootAllocator();
+ VectorSchemaRoot root = VectorSchemaRoot.create(schema, allocator)) {
+ final IntVector ints = (IntVector) root.getVector(0);
+ VectorUnloader unloader = new VectorUnloader(root);
+
+ root.allocateNew();
+ ints.setSafe(0, 1);
+ ints.setSafe(1, 2);
+ ints.setSafe(2, 4);
+ ints.setSafe(3, 8);
+ root.setRowCount(4);
+ batches.add(unloader.getRecordBatch());
+
+ final String exceptionMessage = "This is a message for testing exception.";
+
+ RuntimeException exToThrow = new RuntimeException(exceptionMessage);
+ batches.add(exToThrow);
+
+ StringWriter sw = new StringWriter();
+ PrintWriter pw = new PrintWriter(sw);
+ exToThrow.printStackTrace(pw);
+ final String expectExceptionMessage = sw.toString();
+
+ ArrowReader source = new ExceptionMemoryArrowReader(allocator, schema, batches);
+
+ try (final ArrowArrayStream stream = ArrowArrayStream.allocateNew(allocator);
+ final VectorSchemaRoot importRoot = VectorSchemaRoot.create(schema, allocator)) {
+ final VectorLoader loader = new VectorLoader(importRoot);
+ Data.exportArrayStream(allocator, source, stream);
+
+ Throwable exceptionThrowed = null;
+ try (final ArrowReader reader = Data.importArrayStream(allocator, stream)) {
+ assertThat(reader.getVectorSchemaRoot().getSchema()).isEqualTo(schema);
+
+ for (Object batch : batches) {
+ try {
+ reader.loadNextBatch();
+ } catch (Exception e) {
+ assertThat(exceptionThrowed).isEqualTo(null);
+ final String eMessage = e.getMessage();
+ // 1 for '}', ref to CDataJniException
+ assertThat(eMessage.length()).isGreaterThan(expectExceptionMessage.length() + 1);
+ assertThat(eMessage.substring(eMessage.length() - expectExceptionMessage.length() - 1, eMessage.length() - 1))
+ .isEqualTo(expectExceptionMessage);
Review Comment:
...then you just want endsWith?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix get length of byte[] in TryCopyLastError [arrow-java]
Posted by "hnwyllmm (via GitHub)" <gi...@apache.org>.
hnwyllmm commented on PR #760:
URL: https://github.com/apache/arrow-java/pull/760#issuecomment-2905095542
> > The error message in CI doesn't seem to be related to my PR:
> > ```
> > /opt/rh/devtoolset-10/root/usr/libexec/gcc/x86_64-redhat-linux/10/ld: cannot find -lutf8proc
> > collect2: error: ld returned 1 exit status
> > ```
> >
> >
> > linkage: https://github.com/apache/arrow-java/actions/runs/15140171347/job/42594091462?pr=760
> > Do you have some suggestions?
>
> Could you open a separated issue for it? Let's fix it as a separated task.
issue: #767
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-759: Get length of byte[] in TryCopyLastError [arrow-java]
Posted by "hnwyllmm (via GitHub)" <gi...@apache.org>.
hnwyllmm commented on PR #760:
URL: https://github.com/apache/arrow-java/pull/760#issuecomment-3017879240
Thanks for your patient. It seems the failed CI test cases are not related with this PR now. Please take a look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-759: Get length of byte[] in TryCopyLastError [arrow-java]
Posted by "hnwyllmm (via GitHub)" <gi...@apache.org>.
hnwyllmm commented on code in PR #760:
URL: https://github.com/apache/arrow-java/pull/760#discussion_r2133803749
##########
c/src/test/java/org/apache/arrow/c/ExceptionTest.java:
##########
@@ -0,0 +1,184 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.arrow.c;
+
+import static org.assertj.core.api.Assertions.assertThat;
+
+import java.io.IOException;
+import java.io.PrintWriter;
+import java.io.StringWriter;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.Objects;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+import org.apache.arrow.c.jni.CDataJniException;
+import org.apache.arrow.memory.BufferAllocator;
+import org.apache.arrow.memory.RootAllocator;
+import org.apache.arrow.vector.IntVector;
+import org.apache.arrow.vector.VectorLoader;
+import org.apache.arrow.vector.VectorSchemaRoot;
+import org.apache.arrow.vector.VectorUnloader;
+import org.apache.arrow.vector.dictionary.Dictionary;
+import org.apache.arrow.vector.dictionary.DictionaryProvider;
+import org.apache.arrow.vector.ipc.ArrowReader;
+import org.apache.arrow.vector.ipc.message.ArrowRecordBatch;
+import org.apache.arrow.vector.types.pojo.ArrowType;
+import org.apache.arrow.vector.types.pojo.Field;
+import org.apache.arrow.vector.types.pojo.Schema;
+import org.junit.jupiter.api.Test;
+
+// Regression test for https://github.com/apache/arrow-java/issues/759
+final class ExceptionTest {
+ @Test
+ public void testException() throws IOException {
+ final Schema schema =
+ new Schema(Collections.singletonList(Field.nullable("ints", new ArrowType.Int(32, true))));
+ final List<Object> batches = new ArrayList<>();
+
+ try (BufferAllocator allocator = new RootAllocator();
+ VectorSchemaRoot root = VectorSchemaRoot.create(schema, allocator)) {
+ final IntVector ints = (IntVector) root.getVector(0);
+ VectorUnloader unloader = new VectorUnloader(root);
+
+ root.allocateNew();
+ ints.setSafe(0, 1);
+ ints.setSafe(1, 2);
+ ints.setSafe(2, 4);
+ ints.setSafe(3, 8);
+ root.setRowCount(4);
+ batches.add(unloader.getRecordBatch());
+
+ final String exceptionMessage = "This is a message for testing exception.";
+
+ RuntimeException exToThrow = new RuntimeException(exceptionMessage);
+ batches.add(exToThrow);
+
+ StringWriter sw = new StringWriter();
+ PrintWriter pw = new PrintWriter(sw);
+ exToThrow.printStackTrace(pw);
+ final String expectExceptionMessage = sw.toString();
+
+ ArrowReader source = new ExceptionMemoryArrowReader(allocator, schema, batches);
+
+ try (final ArrowArrayStream stream = ArrowArrayStream.allocateNew(allocator);
+ final VectorSchemaRoot importRoot = VectorSchemaRoot.create(schema, allocator)) {
+ final VectorLoader loader = new VectorLoader(importRoot);
+ Data.exportArrayStream(allocator, source, stream);
+
+ Throwable exceptionThrowed = null;
+ try (final ArrowReader reader = Data.importArrayStream(allocator, stream)) {
+ assertThat(reader.getVectorSchemaRoot().getSchema()).isEqualTo(schema);
+
+ for (Object batch : batches) {
+ try {
+ reader.loadNextBatch();
+ } catch (Exception e) {
+ assertThat(exceptionThrowed).isEqualTo(null);
+ final String eMessage = e.getMessage();
+ // 1 for '}', ref to CDataJniException
+ assertThat(eMessage.length()).isGreaterThan(expectExceptionMessage.length() + 1);
+ assertThat(eMessage.substring(eMessage.length() - expectExceptionMessage.length() - 1, eMessage.length() - 1))
+ .isEqualTo(expectExceptionMessage);
Review Comment:
No, we must make sure the string in the exception is the same with we throws.
"abc\x037" contains "abc", but it's not correct and we want to catch the bug.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix get length of byte[] in TryCopyLastError [arrow-java]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on PR #760:
URL: https://github.com/apache/arrow-java/pull/760#issuecomment-2906090380
Thanks. (We've fixed this problem in apache/arrow. Sorry for not sharing it here...)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-759: Get length of byte[] in TryCopyLastError [arrow-java]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on PR #760:
URL: https://github.com/apache/arrow-java/pull/760#issuecomment-2994640852
Regardless, there are Checkstyle failures
```
Error: /build/c/src/test/java/org/apache/arrow/c/ExceptionTest.java:30:8: Unused import: java.util.Objects. [UnusedImports]
Error: /build/c/src/test/java/org/apache/arrow/c/ExceptionTest.java:33:8: Unused import: org.apache.arrow.c.jni.CDataJniException. [UnusedImports]
Error: /build/c/src/test/java/org/apache/arrow/c/ExceptionTest.java:36:8: Unused import: org.apache.arrow.vector.IntVector. [UnusedImports]
Error: /build/c/src/test/java/org/apache/arrow/c/ExceptionTest.java:39:8: Unused import: org.apache.arrow.vector.VectorUnloader. [UnusedImports]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-759: Get length of byte[] in TryCopyLastError [arrow-java]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm merged PR #760:
URL: https://github.com/apache/arrow-java/pull/760
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-759: Get length of byte[] in TryCopyLastError [arrow-java]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #760:
URL: https://github.com/apache/arrow-java/pull/760#issuecomment-2907736566
<!-- label_helper_comment -->
Thank you for opening a pull request!
Please label the PR with one or more of:
- bug-fix
- chore
- dependencies
- documentation
- enhancement
Also, add the 'breaking-change' label if appropriate.
See [CONTRIBUTING.md](https://github.com/apache/arrow-java/blob/main/CONTRIBUTING.md) for details.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-759: Get length of byte[] in TryCopyLastError [arrow-java]
Posted by "hnwyllmm (via GitHub)" <gi...@apache.org>.
hnwyllmm commented on code in PR #760:
URL: https://github.com/apache/arrow-java/pull/760#discussion_r2133859931
##########
c/src/test/java/org/apache/arrow/c/ExceptionTest.java:
##########
@@ -0,0 +1,184 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.arrow.c;
+
+import static org.assertj.core.api.Assertions.assertThat;
+
+import java.io.IOException;
+import java.io.PrintWriter;
+import java.io.StringWriter;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.Objects;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+import org.apache.arrow.c.jni.CDataJniException;
+import org.apache.arrow.memory.BufferAllocator;
+import org.apache.arrow.memory.RootAllocator;
+import org.apache.arrow.vector.IntVector;
+import org.apache.arrow.vector.VectorLoader;
+import org.apache.arrow.vector.VectorSchemaRoot;
+import org.apache.arrow.vector.VectorUnloader;
+import org.apache.arrow.vector.dictionary.Dictionary;
+import org.apache.arrow.vector.dictionary.DictionaryProvider;
+import org.apache.arrow.vector.ipc.ArrowReader;
+import org.apache.arrow.vector.ipc.message.ArrowRecordBatch;
+import org.apache.arrow.vector.types.pojo.ArrowType;
+import org.apache.arrow.vector.types.pojo.Field;
+import org.apache.arrow.vector.types.pojo.Schema;
+import org.junit.jupiter.api.Test;
+
+// Regression test for https://github.com/apache/arrow-java/issues/759
+final class ExceptionTest {
+ @Test
+ public void testException() throws IOException {
+ final Schema schema =
+ new Schema(Collections.singletonList(Field.nullable("ints", new ArrowType.Int(32, true))));
+ final List<Object> batches = new ArrayList<>();
+
+ try (BufferAllocator allocator = new RootAllocator();
+ VectorSchemaRoot root = VectorSchemaRoot.create(schema, allocator)) {
+ final IntVector ints = (IntVector) root.getVector(0);
+ VectorUnloader unloader = new VectorUnloader(root);
+
+ root.allocateNew();
+ ints.setSafe(0, 1);
+ ints.setSafe(1, 2);
+ ints.setSafe(2, 4);
+ ints.setSafe(3, 8);
+ root.setRowCount(4);
+ batches.add(unloader.getRecordBatch());
+
+ final String exceptionMessage = "This is a message for testing exception.";
+
+ RuntimeException exToThrow = new RuntimeException(exceptionMessage);
+ batches.add(exToThrow);
+
+ StringWriter sw = new StringWriter();
+ PrintWriter pw = new PrintWriter(sw);
+ exToThrow.printStackTrace(pw);
+ final String expectExceptionMessage = sw.toString();
+
+ ArrowReader source = new ExceptionMemoryArrowReader(allocator, schema, batches);
+
+ try (final ArrowArrayStream stream = ArrowArrayStream.allocateNew(allocator);
+ final VectorSchemaRoot importRoot = VectorSchemaRoot.create(schema, allocator)) {
+ final VectorLoader loader = new VectorLoader(importRoot);
+ Data.exportArrayStream(allocator, source, stream);
+
+ Throwable exceptionThrowed = null;
+ try (final ArrowReader reader = Data.importArrayStream(allocator, stream)) {
+ assertThat(reader.getVectorSchemaRoot().getSchema()).isEqualTo(schema);
+
+ for (Object batch : batches) {
+ try {
+ reader.loadNextBatch();
+ } catch (Exception e) {
+ assertThat(exceptionThrowed).isEqualTo(null);
+ final String eMessage = e.getMessage();
+ // 1 for '}', ref to CDataJniException
+ assertThat(eMessage.length()).isGreaterThan(expectExceptionMessage.length() + 1);
+ assertThat(eMessage.substring(eMessage.length() - expectExceptionMessage.length() - 1, eMessage.length() - 1))
+ .isEqualTo(expectExceptionMessage);
Review Comment:
No.
Here is the CDataJniException::getMessage:
```java
public String getMessage() {
return "CDataJniException{errno=" + this.errno + ", message=" + super.getMessage() + '}';
}
```
The message of the exception we thrown is `super().getMessage()`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-759: Get length of byte[] in TryCopyLastError [arrow-java]
Posted by "hnwyllmm (via GitHub)" <gi...@apache.org>.
hnwyllmm commented on code in PR #760:
URL: https://github.com/apache/arrow-java/pull/760#discussion_r2120532636
##########
c/src/test/java/org/apache/arrow/c/ExceptionTest.java:
##########
@@ -0,0 +1,172 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.arrow.c;
+
+import static org.assertj.core.api.Assertions.assertThat;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+import org.apache.arrow.memory.BufferAllocator;
+import org.apache.arrow.memory.RootAllocator;
+import org.apache.arrow.vector.IntVector;
+import org.apache.arrow.vector.VectorLoader;
+import org.apache.arrow.vector.VectorSchemaRoot;
+import org.apache.arrow.vector.VectorUnloader;
+import org.apache.arrow.vector.dictionary.Dictionary;
+import org.apache.arrow.vector.dictionary.DictionaryProvider;
+import org.apache.arrow.vector.ipc.ArrowReader;
+import org.apache.arrow.vector.ipc.message.ArrowRecordBatch;
+import org.apache.arrow.vector.types.pojo.ArrowType;
+import org.apache.arrow.vector.types.pojo.Field;
+import org.apache.arrow.vector.types.pojo.Schema;
+import org.junit.jupiter.api.Test;
+
+final class ExceptionTest {
+ @Test
+ public void testException() throws IOException {
+ final Schema schema =
+ new Schema(Collections.singletonList(Field.nullable("ints", new ArrowType.Int(32, true))));
+ final List<Object> batches = new ArrayList<>();
+
+ try (BufferAllocator allocator = new RootAllocator();
+ VectorSchemaRoot root = VectorSchemaRoot.create(schema, allocator)) {
+ final IntVector ints = (IntVector) root.getVector(0);
+ VectorUnloader unloader = new VectorUnloader(root);
+
+ root.allocateNew();
+ ints.setSafe(0, 1);
+ ints.setSafe(1, 2);
+ ints.setSafe(2, 4);
+ ints.setSafe(3, 8);
+ root.setRowCount(4);
+ batches.add(unloader.getRecordBatch());
+
+ final String exceptionMessage = "java.lang.RuntimeException: Error occurred while getting next schema root.\n\tat org.apache.arrow.adapter.jdbc.ArrowVectorIterator.next(ArrowVectorIterator.java:205)\n\tat com.oceanbase.external.jdbc.JdbcScanner.loadNextBatch(JdbcScanner.java:73)\n\tat org.apache.arrow.c.ArrayStreamExporter$ExportedArrayStreamPrivateData.getNext(ArrayStreamExporter.java:72)\nCaused by: java.lang.RuntimeException: Error occurred while consuming data.\n\tat org.apache.arrow.adapter.jdbc.ArrowVectorIterator.consumeData(ArrowVectorIterator.java:127)\n\tat org.apache.arrow.adapter.jdbc.ArrowVectorIterator.load(ArrowVectorIterator.java:178)\n\tat org.apache.arrow.adapter.jdbc.ArrowVectorIterator.next(ArrowVectorIterator.java:198)\n\t... 2 more\nCaused by: java.lang.OutOfMemoryError: Java heap space\n";
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix get length of byte[] in TryCopyLastError [arrow-java]
Posted by "hnwyllmm (via GitHub)" <gi...@apache.org>.
hnwyllmm commented on PR #760:
URL: https://github.com/apache/arrow-java/pull/760#issuecomment-2907712247
> > > Is it possible to add a test?
> >
> >
> > I'll try to add a test case in `integration_tests.py`.
>
> Do you need an integration test? IMO, you should be able to export a stream, then immediately import it, all within Java.
I add a unittest but I think it can't make sure `TryCopyLastError` works correctly, so I suggest it should be removed.
We can read memory after the error string address in most cases. We usually check such problems by sanity tools.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-759: Get length of byte[] in TryCopyLastError [arrow-java]
Posted by "hnwyllmm (via GitHub)" <gi...@apache.org>.
hnwyllmm commented on code in PR #760:
URL: https://github.com/apache/arrow-java/pull/760#discussion_r2120516670
##########
c/src/test/java/org/apache/arrow/c/ExceptionTest.java:
##########
@@ -0,0 +1,172 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.arrow.c;
+
+import static org.assertj.core.api.Assertions.assertThat;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+import org.apache.arrow.memory.BufferAllocator;
+import org.apache.arrow.memory.RootAllocator;
+import org.apache.arrow.vector.IntVector;
+import org.apache.arrow.vector.VectorLoader;
+import org.apache.arrow.vector.VectorSchemaRoot;
+import org.apache.arrow.vector.VectorUnloader;
+import org.apache.arrow.vector.dictionary.Dictionary;
+import org.apache.arrow.vector.dictionary.DictionaryProvider;
+import org.apache.arrow.vector.ipc.ArrowReader;
+import org.apache.arrow.vector.ipc.message.ArrowRecordBatch;
+import org.apache.arrow.vector.types.pojo.ArrowType;
+import org.apache.arrow.vector.types.pojo.Field;
+import org.apache.arrow.vector.types.pojo.Schema;
+import org.junit.jupiter.api.Test;
+
+final class ExceptionTest {
+ @Test
+ public void testException() throws IOException {
Review Comment:
DONE
##########
c/src/test/java/org/apache/arrow/c/ExceptionTest.java:
##########
@@ -0,0 +1,172 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.arrow.c;
+
+import static org.assertj.core.api.Assertions.assertThat;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+import org.apache.arrow.memory.BufferAllocator;
+import org.apache.arrow.memory.RootAllocator;
+import org.apache.arrow.vector.IntVector;
+import org.apache.arrow.vector.VectorLoader;
+import org.apache.arrow.vector.VectorSchemaRoot;
+import org.apache.arrow.vector.VectorUnloader;
+import org.apache.arrow.vector.dictionary.Dictionary;
+import org.apache.arrow.vector.dictionary.DictionaryProvider;
+import org.apache.arrow.vector.ipc.ArrowReader;
+import org.apache.arrow.vector.ipc.message.ArrowRecordBatch;
+import org.apache.arrow.vector.types.pojo.ArrowType;
+import org.apache.arrow.vector.types.pojo.Field;
+import org.apache.arrow.vector.types.pojo.Schema;
+import org.junit.jupiter.api.Test;
+
+final class ExceptionTest {
+ @Test
+ public void testException() throws IOException {
+ final Schema schema =
+ new Schema(Collections.singletonList(Field.nullable("ints", new ArrowType.Int(32, true))));
+ final List<Object> batches = new ArrayList<>();
+
+ try (BufferAllocator allocator = new RootAllocator();
+ VectorSchemaRoot root = VectorSchemaRoot.create(schema, allocator)) {
+ final IntVector ints = (IntVector) root.getVector(0);
+ VectorUnloader unloader = new VectorUnloader(root);
+
+ root.allocateNew();
+ ints.setSafe(0, 1);
+ ints.setSafe(1, 2);
+ ints.setSafe(2, 4);
+ ints.setSafe(3, 8);
+ root.setRowCount(4);
+ batches.add(unloader.getRecordBatch());
+
+ final String exceptionMessage = "java.lang.RuntimeException: Error occurred while getting next schema root.\n\tat org.apache.arrow.adapter.jdbc.ArrowVectorIterator.next(ArrowVectorIterator.java:205)\n\tat com.oceanbase.external.jdbc.JdbcScanner.loadNextBatch(JdbcScanner.java:73)\n\tat org.apache.arrow.c.ArrayStreamExporter$ExportedArrayStreamPrivateData.getNext(ArrayStreamExporter.java:72)\nCaused by: java.lang.RuntimeException: Error occurred while consuming data.\n\tat org.apache.arrow.adapter.jdbc.ArrowVectorIterator.consumeData(ArrowVectorIterator.java:127)\n\tat org.apache.arrow.adapter.jdbc.ArrowVectorIterator.load(ArrowVectorIterator.java:178)\n\tat org.apache.arrow.adapter.jdbc.ArrowVectorIterator.next(ArrowVectorIterator.java:198)\n\t... 2 more\nCaused by: java.lang.OutOfMemoryError: Java heap space\n";
+
+ RuntimeException ex = new RuntimeException(exceptionMessage);
+ batches.add(ex);
+
+ ArrowReader source = new ExceptionMemoryArrowReader(allocator, schema, batches);
+
+ try (final ArrowArrayStream stream = ArrowArrayStream.allocateNew(allocator);
+ final VectorSchemaRoot importRoot = VectorSchemaRoot.create(schema, allocator)) {
+ final VectorLoader loader = new VectorLoader(importRoot);
+ Data.exportArrayStream(allocator, source, stream);
+
+ try (final ArrowReader reader = Data.importArrayStream(allocator, stream)) {
+ assertThat(reader.getVectorSchemaRoot().getSchema()).isEqualTo(schema);
+
+ for (Object batch : batches) {
+ try {
+ reader.loadNextBatch();
+ } catch (Exception e) {
+ Throwable exToCheck = e;
+ while (exToCheck != null && !(exToCheck instanceof RuntimeException)) {
+ exToCheck = exToCheck.getCause();
+ }
+ if (exToCheck != null) {
+ assertThat(exToCheck.getMessage()).isEqualTo(exceptionMessage);
+ }
+ continue;
+ }
+ loader.load((ArrowRecordBatch) batch);
+
+ assertThat(reader.getVectorSchemaRoot().getRowCount()).isEqualTo(root.getRowCount());
+ }
+ }
+ }
+ }
Review Comment:
DONE
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-759: Get length of byte[] in TryCopyLastError [arrow-java]
Posted by "hnwyllmm (via GitHub)" <gi...@apache.org>.
hnwyllmm commented on PR #760:
URL: https://github.com/apache/arrow-java/pull/760#issuecomment-3017632459
> Regardless, there are Checkstyle failures
>
> ```
>
> Error: /build/c/src/test/java/org/apache/arrow/c/ExceptionTest.java:30:8: Unused import: java.util.Objects. [UnusedImports]
> Error: /build/c/src/test/java/org/apache/arrow/c/ExceptionTest.java:33:8: Unused import: org.apache.arrow.c.jni.CDataJniException. [UnusedImports]
> Error: /build/c/src/test/java/org/apache/arrow/c/ExceptionTest.java:36:8: Unused import: org.apache.arrow.vector.IntVector. [UnusedImports]
> Error: /build/c/src/test/java/org/apache/arrow/c/ExceptionTest.java:39:8: Unused import: org.apache.arrow.vector.VectorUnloader. [UnusedImports]
> ```
sorry for that. fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-759: Get length of byte[] in TryCopyLastError [arrow-java]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on code in PR #760:
URL: https://github.com/apache/arrow-java/pull/760#discussion_r2125296888
##########
c/src/test/java/org/apache/arrow/c/ExceptionTest.java:
##########
@@ -0,0 +1,184 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.arrow.c;
+
+import static org.assertj.core.api.Assertions.assertThat;
+
+import java.io.IOException;
+import java.io.PrintWriter;
+import java.io.StringWriter;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.Objects;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+import org.apache.arrow.c.jni.CDataJniException;
+import org.apache.arrow.memory.BufferAllocator;
+import org.apache.arrow.memory.RootAllocator;
+import org.apache.arrow.vector.IntVector;
+import org.apache.arrow.vector.VectorLoader;
+import org.apache.arrow.vector.VectorSchemaRoot;
+import org.apache.arrow.vector.VectorUnloader;
+import org.apache.arrow.vector.dictionary.Dictionary;
+import org.apache.arrow.vector.dictionary.DictionaryProvider;
+import org.apache.arrow.vector.ipc.ArrowReader;
+import org.apache.arrow.vector.ipc.message.ArrowRecordBatch;
+import org.apache.arrow.vector.types.pojo.ArrowType;
+import org.apache.arrow.vector.types.pojo.Field;
+import org.apache.arrow.vector.types.pojo.Schema;
+import org.junit.jupiter.api.Test;
+
+// Regression test for https://github.com/apache/arrow-java/issues/759
+final class ExceptionTest {
+ @Test
+ public void testException() throws IOException {
+ final Schema schema =
+ new Schema(Collections.singletonList(Field.nullable("ints", new ArrowType.Int(32, true))));
+ final List<Object> batches = new ArrayList<>();
+
+ try (BufferAllocator allocator = new RootAllocator();
+ VectorSchemaRoot root = VectorSchemaRoot.create(schema, allocator)) {
+ final IntVector ints = (IntVector) root.getVector(0);
+ VectorUnloader unloader = new VectorUnloader(root);
+
+ root.allocateNew();
+ ints.setSafe(0, 1);
+ ints.setSafe(1, 2);
+ ints.setSafe(2, 4);
+ ints.setSafe(3, 8);
+ root.setRowCount(4);
+ batches.add(unloader.getRecordBatch());
+
+ final String exceptionMessage = "This is a message for testing exception.";
+
+ RuntimeException exToThrow = new RuntimeException(exceptionMessage);
+ batches.add(exToThrow);
+
+ StringWriter sw = new StringWriter();
+ PrintWriter pw = new PrintWriter(sw);
+ exToThrow.printStackTrace(pw);
+ final String expectExceptionMessage = sw.toString();
+
+ ArrowReader source = new ExceptionMemoryArrowReader(allocator, schema, batches);
+
+ try (final ArrowArrayStream stream = ArrowArrayStream.allocateNew(allocator);
+ final VectorSchemaRoot importRoot = VectorSchemaRoot.create(schema, allocator)) {
+ final VectorLoader loader = new VectorLoader(importRoot);
+ Data.exportArrayStream(allocator, source, stream);
+
+ Throwable exceptionThrowed = null;
+ try (final ArrowReader reader = Data.importArrayStream(allocator, stream)) {
+ assertThat(reader.getVectorSchemaRoot().getSchema()).isEqualTo(schema);
+
+ for (Object batch : batches) {
+ try {
+ reader.loadNextBatch();
+ } catch (Exception e) {
+ assertThat(exceptionThrowed).isEqualTo(null);
+ final String eMessage = e.getMessage();
+ // 1 for '}', ref to CDataJniException
+ assertThat(eMessage.length()).isGreaterThan(expectExceptionMessage.length() + 1);
+ assertThat(eMessage.substring(eMessage.length() - expectExceptionMessage.length() - 1, eMessage.length() - 1))
+ .isEqualTo(expectExceptionMessage);
Review Comment:
Just use `contains`...
##########
c/src/test/java/org/apache/arrow/c/ExceptionTest.java:
##########
@@ -0,0 +1,184 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.arrow.c;
+
+import static org.assertj.core.api.Assertions.assertThat;
+
+import java.io.IOException;
+import java.io.PrintWriter;
+import java.io.StringWriter;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.Objects;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+import org.apache.arrow.c.jni.CDataJniException;
+import org.apache.arrow.memory.BufferAllocator;
+import org.apache.arrow.memory.RootAllocator;
+import org.apache.arrow.vector.IntVector;
+import org.apache.arrow.vector.VectorLoader;
+import org.apache.arrow.vector.VectorSchemaRoot;
+import org.apache.arrow.vector.VectorUnloader;
+import org.apache.arrow.vector.dictionary.Dictionary;
+import org.apache.arrow.vector.dictionary.DictionaryProvider;
+import org.apache.arrow.vector.ipc.ArrowReader;
+import org.apache.arrow.vector.ipc.message.ArrowRecordBatch;
+import org.apache.arrow.vector.types.pojo.ArrowType;
+import org.apache.arrow.vector.types.pojo.Field;
+import org.apache.arrow.vector.types.pojo.Schema;
+import org.junit.jupiter.api.Test;
+
+// Regression test for https://github.com/apache/arrow-java/issues/759
+final class ExceptionTest {
+ @Test
+ public void testException() throws IOException {
+ final Schema schema =
+ new Schema(Collections.singletonList(Field.nullable("ints", new ArrowType.Int(32, true))));
+ final List<Object> batches = new ArrayList<>();
+
+ try (BufferAllocator allocator = new RootAllocator();
+ VectorSchemaRoot root = VectorSchemaRoot.create(schema, allocator)) {
+ final IntVector ints = (IntVector) root.getVector(0);
+ VectorUnloader unloader = new VectorUnloader(root);
+
+ root.allocateNew();
+ ints.setSafe(0, 1);
+ ints.setSafe(1, 2);
+ ints.setSafe(2, 4);
+ ints.setSafe(3, 8);
+ root.setRowCount(4);
+ batches.add(unloader.getRecordBatch());
+
+ final String exceptionMessage = "This is a message for testing exception.";
+
+ RuntimeException exToThrow = new RuntimeException(exceptionMessage);
+ batches.add(exToThrow);
+
+ StringWriter sw = new StringWriter();
+ PrintWriter pw = new PrintWriter(sw);
+ exToThrow.printStackTrace(pw);
+ final String expectExceptionMessage = sw.toString();
+
+ ArrowReader source = new ExceptionMemoryArrowReader(allocator, schema, batches);
+
+ try (final ArrowArrayStream stream = ArrowArrayStream.allocateNew(allocator);
+ final VectorSchemaRoot importRoot = VectorSchemaRoot.create(schema, allocator)) {
+ final VectorLoader loader = new VectorLoader(importRoot);
+ Data.exportArrayStream(allocator, source, stream);
+
+ Throwable exceptionThrowed = null;
+ try (final ArrowReader reader = Data.importArrayStream(allocator, stream)) {
+ assertThat(reader.getVectorSchemaRoot().getSchema()).isEqualTo(schema);
+
+ for (Object batch : batches) {
+ try {
+ reader.loadNextBatch();
+ } catch (Exception e) {
+ assertThat(exceptionThrowed).isEqualTo(null);
+ final String eMessage = e.getMessage();
+ // 1 for '}', ref to CDataJniException
+ assertThat(eMessage.length()).isGreaterThan(expectExceptionMessage.length() + 1);
+ assertThat(eMessage.substring(eMessage.length() - expectExceptionMessage.length() - 1, eMessage.length() - 1))
+ .isEqualTo(expectExceptionMessage);
+ exceptionThrowed = e;
+ continue;
+ }
Review Comment:
Use assertThrows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-759: Get length of byte[] in TryCopyLastError [arrow-java]
Posted by "hnwyllmm (via GitHub)" <gi...@apache.org>.
hnwyllmm commented on PR #760:
URL: https://github.com/apache/arrow-java/pull/760#issuecomment-2924777854
> Can't the test validate that the exception has the right error message after making it through the FFI boundary?
PTAL.
I use a real exception message in the ExceptionTest.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-759: Get length of byte[] in TryCopyLastError [arrow-java]
Posted by "hnwyllmm (via GitHub)" <gi...@apache.org>.
hnwyllmm commented on code in PR #760:
URL: https://github.com/apache/arrow-java/pull/760#discussion_r2133859931
##########
c/src/test/java/org/apache/arrow/c/ExceptionTest.java:
##########
@@ -0,0 +1,184 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.arrow.c;
+
+import static org.assertj.core.api.Assertions.assertThat;
+
+import java.io.IOException;
+import java.io.PrintWriter;
+import java.io.StringWriter;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.Objects;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+import org.apache.arrow.c.jni.CDataJniException;
+import org.apache.arrow.memory.BufferAllocator;
+import org.apache.arrow.memory.RootAllocator;
+import org.apache.arrow.vector.IntVector;
+import org.apache.arrow.vector.VectorLoader;
+import org.apache.arrow.vector.VectorSchemaRoot;
+import org.apache.arrow.vector.VectorUnloader;
+import org.apache.arrow.vector.dictionary.Dictionary;
+import org.apache.arrow.vector.dictionary.DictionaryProvider;
+import org.apache.arrow.vector.ipc.ArrowReader;
+import org.apache.arrow.vector.ipc.message.ArrowRecordBatch;
+import org.apache.arrow.vector.types.pojo.ArrowType;
+import org.apache.arrow.vector.types.pojo.Field;
+import org.apache.arrow.vector.types.pojo.Schema;
+import org.junit.jupiter.api.Test;
+
+// Regression test for https://github.com/apache/arrow-java/issues/759
+final class ExceptionTest {
+ @Test
+ public void testException() throws IOException {
+ final Schema schema =
+ new Schema(Collections.singletonList(Field.nullable("ints", new ArrowType.Int(32, true))));
+ final List<Object> batches = new ArrayList<>();
+
+ try (BufferAllocator allocator = new RootAllocator();
+ VectorSchemaRoot root = VectorSchemaRoot.create(schema, allocator)) {
+ final IntVector ints = (IntVector) root.getVector(0);
+ VectorUnloader unloader = new VectorUnloader(root);
+
+ root.allocateNew();
+ ints.setSafe(0, 1);
+ ints.setSafe(1, 2);
+ ints.setSafe(2, 4);
+ ints.setSafe(3, 8);
+ root.setRowCount(4);
+ batches.add(unloader.getRecordBatch());
+
+ final String exceptionMessage = "This is a message for testing exception.";
+
+ RuntimeException exToThrow = new RuntimeException(exceptionMessage);
+ batches.add(exToThrow);
+
+ StringWriter sw = new StringWriter();
+ PrintWriter pw = new PrintWriter(sw);
+ exToThrow.printStackTrace(pw);
+ final String expectExceptionMessage = sw.toString();
+
+ ArrowReader source = new ExceptionMemoryArrowReader(allocator, schema, batches);
+
+ try (final ArrowArrayStream stream = ArrowArrayStream.allocateNew(allocator);
+ final VectorSchemaRoot importRoot = VectorSchemaRoot.create(schema, allocator)) {
+ final VectorLoader loader = new VectorLoader(importRoot);
+ Data.exportArrayStream(allocator, source, stream);
+
+ Throwable exceptionThrowed = null;
+ try (final ArrowReader reader = Data.importArrayStream(allocator, stream)) {
+ assertThat(reader.getVectorSchemaRoot().getSchema()).isEqualTo(schema);
+
+ for (Object batch : batches) {
+ try {
+ reader.loadNextBatch();
+ } catch (Exception e) {
+ assertThat(exceptionThrowed).isEqualTo(null);
+ final String eMessage = e.getMessage();
+ // 1 for '}', ref to CDataJniException
+ assertThat(eMessage.length()).isGreaterThan(expectExceptionMessage.length() + 1);
+ assertThat(eMessage.substring(eMessage.length() - expectExceptionMessage.length() - 1, eMessage.length() - 1))
+ .isEqualTo(expectExceptionMessage);
Review Comment:
I use `endsWith` to simplified the code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix get length of byte[] in TryCopyLastError [arrow-java]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on PR #760:
URL: https://github.com/apache/arrow-java/pull/760#issuecomment-2896399246
> > Is it possible to add a test?
>
> I'll try to add a test case in `integration_tests.py`.
Do you need an integration test? IMO, you should be able to export a stream, then immediately import it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix get length of byte[] in TryCopyLastError [arrow-java]
Posted by "hnwyllmm (via GitHub)" <gi...@apache.org>.
hnwyllmm commented on PR #760:
URL: https://github.com/apache/arrow-java/pull/760#issuecomment-2896238789
> Is it possible to add a test?
I'll try to add a test case in `integration_tests.py`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix get length of byte[] in TryCopyLastError [arrow-java]
Posted by "hnwyllmm (via GitHub)" <gi...@apache.org>.
hnwyllmm commented on PR #760:
URL: https://github.com/apache/arrow-java/pull/760#issuecomment-2896233817
The error message in CI doesn't seem to be related to my PR:
```text
/opt/rh/devtoolset-10/root/usr/libexec/gcc/x86_64-redhat-linux/10/ld: cannot find -lutf8proc
collect2: error: ld returned 1 exit status
```
linkage: https://github.com/apache/arrow-java/actions/runs/15140171347/job/42594091462?pr=760
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-759: Get length of byte[] in TryCopyLastError [arrow-java]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on PR #760:
URL: https://github.com/apache/arrow-java/pull/760#issuecomment-2907736631
Can't the test validate that the exception has the right error message after making it through the FFI boundary?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-759: Get length of byte[] in TryCopyLastError [arrow-java]
Posted by "hnwyllmm (via GitHub)" <gi...@apache.org>.
hnwyllmm commented on code in PR #760:
URL: https://github.com/apache/arrow-java/pull/760#discussion_r2133804244
##########
c/src/test/java/org/apache/arrow/c/ExceptionTest.java:
##########
@@ -0,0 +1,184 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.arrow.c;
+
+import static org.assertj.core.api.Assertions.assertThat;
+
+import java.io.IOException;
+import java.io.PrintWriter;
+import java.io.StringWriter;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.Objects;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+import org.apache.arrow.c.jni.CDataJniException;
+import org.apache.arrow.memory.BufferAllocator;
+import org.apache.arrow.memory.RootAllocator;
+import org.apache.arrow.vector.IntVector;
+import org.apache.arrow.vector.VectorLoader;
+import org.apache.arrow.vector.VectorSchemaRoot;
+import org.apache.arrow.vector.VectorUnloader;
+import org.apache.arrow.vector.dictionary.Dictionary;
+import org.apache.arrow.vector.dictionary.DictionaryProvider;
+import org.apache.arrow.vector.ipc.ArrowReader;
+import org.apache.arrow.vector.ipc.message.ArrowRecordBatch;
+import org.apache.arrow.vector.types.pojo.ArrowType;
+import org.apache.arrow.vector.types.pojo.Field;
+import org.apache.arrow.vector.types.pojo.Schema;
+import org.junit.jupiter.api.Test;
+
+// Regression test for https://github.com/apache/arrow-java/issues/759
+final class ExceptionTest {
+ @Test
+ public void testException() throws IOException {
+ final Schema schema =
+ new Schema(Collections.singletonList(Field.nullable("ints", new ArrowType.Int(32, true))));
+ final List<Object> batches = new ArrayList<>();
+
+ try (BufferAllocator allocator = new RootAllocator();
+ VectorSchemaRoot root = VectorSchemaRoot.create(schema, allocator)) {
+ final IntVector ints = (IntVector) root.getVector(0);
+ VectorUnloader unloader = new VectorUnloader(root);
+
+ root.allocateNew();
+ ints.setSafe(0, 1);
+ ints.setSafe(1, 2);
+ ints.setSafe(2, 4);
+ ints.setSafe(3, 8);
+ root.setRowCount(4);
+ batches.add(unloader.getRecordBatch());
+
+ final String exceptionMessage = "This is a message for testing exception.";
+
+ RuntimeException exToThrow = new RuntimeException(exceptionMessage);
+ batches.add(exToThrow);
+
+ StringWriter sw = new StringWriter();
+ PrintWriter pw = new PrintWriter(sw);
+ exToThrow.printStackTrace(pw);
+ final String expectExceptionMessage = sw.toString();
+
+ ArrowReader source = new ExceptionMemoryArrowReader(allocator, schema, batches);
+
+ try (final ArrowArrayStream stream = ArrowArrayStream.allocateNew(allocator);
+ final VectorSchemaRoot importRoot = VectorSchemaRoot.create(schema, allocator)) {
+ final VectorLoader loader = new VectorLoader(importRoot);
+ Data.exportArrayStream(allocator, source, stream);
+
+ Throwable exceptionThrowed = null;
+ try (final ArrowReader reader = Data.importArrayStream(allocator, stream)) {
+ assertThat(reader.getVectorSchemaRoot().getSchema()).isEqualTo(schema);
+
+ for (Object batch : batches) {
+ try {
+ reader.loadNextBatch();
+ } catch (Exception e) {
+ assertThat(exceptionThrowed).isEqualTo(null);
+ final String eMessage = e.getMessage();
+ // 1 for '}', ref to CDataJniException
+ assertThat(eMessage.length()).isGreaterThan(expectExceptionMessage.length() + 1);
+ assertThat(eMessage.substring(eMessage.length() - expectExceptionMessage.length() - 1, eMessage.length() - 1))
+ .isEqualTo(expectExceptionMessage);
+ exceptionThrowed = e;
+ continue;
+ }
Review Comment:
DONE.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org