You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Micah Kornfield <em...@gmail.com> on 2021/08/11 22:24:25 UTC
[Rust] Integration tests for recursive nested data?
One of my PRs is showing integration test failures with Rust [1] for the
recursive nested test.
With an error:
Validating
/tmp/tmpg3zo83yh/de3ef975_generated_recursive_nested.json_as_file and
/tmp/arrow-integration-0bm7hcmd/generated_recursive_nested.json
Schemas match. JSON file has 2 batches.
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value',
arrow/src/array/equal/list.rs:143:18
note: run with `RUST_BACKTRACE=1` environment variable to display a
backtrace
I don't think I did anything that would have touched this code for Rust,
and it seems like Rust is failing to consume for all producing languages:
FAILED TEST: recursive_nested C++ producing, Rust consuming
FAILED TEST: recursive_nested Java producing, Rust consuming
FAILED TEST: recursive_nested JS producing, Rust consuming
FAILED TEST: recursive_nested Rust producing, Rust consuming
Is this a known issue? Anything I can try doing to fix this?
Thanks,
Micah
[1]
https://github.com/apache/arrow/pull/10177/checks?check_run_id=3305682790
Re: [Rust] Integration tests for recursive nested data?
Posted by Jorge Cardoso Leitão <jo...@gmail.com>.
Hi,
The checkout of arrow-rs on the failed build is over fa5acd971c97, which up
to 3hrs or so was master, so, I think it is picking the right code.
Did a quick investigation:
* The integration tests on arrow-rs have not been running since June the
30th. they stopped running after the merge of [1], which broke the workflow
[2].
* the integration tests have been running on apache/arrow - latest run was
2hrs against 1a5c26bc2af73 [3], which, as of writing, is master.
* the integration tests on apache/arrow have an annotation of an error [3]
(step Docker Push, failed authentication)
* Filed https://github.com/apache/arrow-rs/issues/690 for the broken CI on
arrow-rs
* Filed https://issues.apache.org/jira/browse/ARROW-13621 for annotation
error on apache/arrow
Coming to the actual error: given [3], that the interval branch did not
change anything related to nested, and that all other consumers are ok, I
can only conclude that this is a non-deterministic behavior on the arrow-rs
side.
[1]
https://github.com/apache/arrow-rs/commit/31bc052126abc4834edf9b3cd7cb72384f84ba3e
[2] https://github.com/apache/arrow-rs/actions/runs/993286543
[3] https://github.com/apache/arrow/runs/3314893335?check_suite_focus=true
Best,
Jorge
On Thu, Aug 12, 2021 at 12:57 PM Andrew Lamb <al...@influxdata.com> wrote:
> Hi Micah,
>
> There is no open issue that I know of, and while I may be mistaken it looks
> like the most recent run of the Integration Test on apache/master [3]
> passed successfully.
>
> The code [1] that is asserting on your branch doesn't look like it has been
> touched for a while (last touched in March 2021). Also, I reviewed the
> latest changes on arrow-rs master [2] and didn't see any obvious changes to
> the handling of nested structures.
>
> I wonder if it is picking up some old version of the crate or something
>
> Andrew
>
>
> [1]
>
> https://github.com/apache/arrow-rs/blob/fa5acd971c973161f17e69d5c6b50d6e77c7da03/arrow/src/array/equal/list.rs#L143
>
> [2] https://github.com/apache/arrow-rs/commits/master
>
> [3] https://github.com/apache/arrow/runs/3309489410
>
>
> On Wed, Aug 11, 2021 at 5:24 PM Micah Kornfield <em...@gmail.com>
> wrote:
>
> > One of my PRs is showing integration test failures with Rust [1] for the
> > recursive nested test.
> >
> > With an error:
> >
> > Validating
> > /tmp/tmpg3zo83yh/de3ef975_generated_recursive_nested.json_as_file and
> > /tmp/arrow-integration-0bm7hcmd/generated_recursive_nested.json
> > Schemas match. JSON file has 2 batches.
> > thread 'main' panicked at 'called `Option::unwrap()` on a `None` value',
> > arrow/src/array/equal/list.rs:143:18
> > note: run with `RUST_BACKTRACE=1` environment variable to display a
> > backtrace
> >
> >
> > I don't think I did anything that would have touched this code for Rust,
> > and it seems like Rust is failing to consume for all producing languages:
> >
> > FAILED TEST: recursive_nested C++ producing, Rust consuming
> > FAILED TEST: recursive_nested Java producing, Rust consuming
> > FAILED TEST: recursive_nested JS producing, Rust consuming
> > FAILED TEST: recursive_nested Rust producing, Rust consuming
> >
> > Is this a known issue? Anything I can try doing to fix this?
> >
> > Thanks,
> > Micah
> >
> >
> > [1]
> >
> https://github.com/apache/arrow/pull/10177/checks?check_run_id=3305682790
> >
>
Re: [Rust] Integration tests for recursive nested data?
Posted by Andrew Lamb <al...@influxdata.com>.
Hi Micah,
There is no open issue that I know of, and while I may be mistaken it looks
like the most recent run of the Integration Test on apache/master [3]
passed successfully.
The code [1] that is asserting on your branch doesn't look like it has been
touched for a while (last touched in March 2021). Also, I reviewed the
latest changes on arrow-rs master [2] and didn't see any obvious changes to
the handling of nested structures.
I wonder if it is picking up some old version of the crate or something
Andrew
[1]
https://github.com/apache/arrow-rs/blob/fa5acd971c973161f17e69d5c6b50d6e77c7da03/arrow/src/array/equal/list.rs#L143
[2] https://github.com/apache/arrow-rs/commits/master
[3] https://github.com/apache/arrow/runs/3309489410
On Wed, Aug 11, 2021 at 5:24 PM Micah Kornfield <em...@gmail.com>
wrote:
> One of my PRs is showing integration test failures with Rust [1] for the
> recursive nested test.
>
> With an error:
>
> Validating
> /tmp/tmpg3zo83yh/de3ef975_generated_recursive_nested.json_as_file and
> /tmp/arrow-integration-0bm7hcmd/generated_recursive_nested.json
> Schemas match. JSON file has 2 batches.
> thread 'main' panicked at 'called `Option::unwrap()` on a `None` value',
> arrow/src/array/equal/list.rs:143:18
> note: run with `RUST_BACKTRACE=1` environment variable to display a
> backtrace
>
>
> I don't think I did anything that would have touched this code for Rust,
> and it seems like Rust is failing to consume for all producing languages:
>
> FAILED TEST: recursive_nested C++ producing, Rust consuming
> FAILED TEST: recursive_nested Java producing, Rust consuming
> FAILED TEST: recursive_nested JS producing, Rust consuming
> FAILED TEST: recursive_nested Rust producing, Rust consuming
>
> Is this a known issue? Anything I can try doing to fix this?
>
> Thanks,
> Micah
>
>
> [1]
> https://github.com/apache/arrow/pull/10177/checks?check_run_id=3305682790
>