You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "JoaoAparicio (via GitHub)" <gi...@apache.org> on 2023/04/09 00:51:37 UTC
[GitHub] [arrow-julia] JoaoAparicio opened a new issue, #420: Flaky test
JoaoAparicio opened a new issue, #420:
URL: https://github.com/apache/arrow-julia/issues/420
Found a flaky test, this one
https://github.com/apache/arrow-julia/blob/c469151d4ff261b50c59bf98101f068fa577fca4/test/runtests.jl#L297-L305
If you run it without threads, always passes. If you run it with `-t 10` it has a significant chance of erroring.
A) Start Julia with `-t 10`
B) Create the environment with:
```julia-repl
]activate --temp
add Arrow
add Tables
add PooledArrays
```
C) Run the test many times:
```julia
using Tables
using Arrow
using PooledArrays
for _ in 1:10000
t = Tables.partitioner(
(
(a=Arrow.toarrowvector(PooledArray([1,2,3 ])),),
(a=Arrow.toarrowvector(PooledArray([1,2,3,4])),),
(a=Arrow.toarrowvector(PooledArray([1,2,3,4,5])),),
)
)
tt = Arrow.Table(Arrow.tobuffer(t))
end
```
Highly likely that you'll hit this:
```julia
(jl_k3aixG) pkg> ERROR: TaskFailedException
Stacktrace:
[1] wait
@ ./task.jl:349 [inlined]
[2] Arrow.Table(blobs::Vector{Arrow.ArrowBlob}; convert::Bool)
@ Arrow ~/.julia/packages/Arrow/P0wVk/src/table.jl:386
[3] Table
@ ~/.julia/packages/Arrow/P0wVk/src/table.jl:295 [inlined]
[4] Arrow.Table(input::IOBuffer, pos::Int64, len::Nothing; kw::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
@ Arrow ~/.julia/packages/Arrow/P0wVk/src/table.jl:290
[5] Table
@ ~/.julia/packages/Arrow/P0wVk/src/table.jl:290 [inlined]
[6] Arrow.Table(input::IOBuffer)
@ Arrow ~/.julia/packages/Arrow/P0wVk/src/table.jl:290
[7] top-level scope
@ ./Untitled-2:13
nested task error: MethodError: no method matching resize!(::Arrow.DictEncoded{Int64, Int8, Arrow.Primitive{Int64, Vector{Int64}}}, ::Int64)
Closest candidates are:
resize!(::Vector, ::Integer)
@ Base array.jl:1246
resize!(::BitVector, ::Integer)
@ Base bitarray.jl:814
resize!(::SparseArrays.ReadOnly, ::Any)
@ SparseArrays ~/julia-1.9.0-rc2/share/julia/stdlib/v1.9/SparseArrays/src/readonly.jl:33
...
Stacktrace:
[1] _append!(a::Arrow.DictEncoded{Int64, Int8, Arrow.Primitive{Int64, Vector{Int64}}}, #unused#::Base.HasLength, iter::Tuple{Int64})
@ Base ./array.jl:1134
[2] append!(a::Arrow.DictEncoded{Int64, Int8, Arrow.Primitive{Int64, Vector{Int64}}}, iter::Tuple{Int64})
@ Base ./array.jl:1126
[3] push!(a::Arrow.DictEncoded{Int64, Int8, Arrow.Primitive{Int64, Vector{Int64}}}, iter::Int64)
@ Base ./array.jl:1127
[4] push!(A::SentinelArrays.ChainedVector{Int64, Arrow.DictEncoded{Int64, Int8, Arrow.Primitive{Int64, Vector{Int64}}}}, val::Int64)
@ SentinelArrays ~/.julia/packages/SentinelArrays/BcfVF/src/chainedvector.jl:506
[5] append!(A::SentinelArrays.ChainedVector{Int64, Arrow.DictEncoded{Int64, Int8, Arrow.Primitive{Int64, Vector{Int64}}}}, B::Arrow.DictEncoded{Int64, Int8, SentinelArrays.ChainedVector{Int64, Arrow.Primitive{Int64, Vector{Int64}}}})
@ SentinelArrays ~/.julia/packages/SentinelArrays/BcfVF/src/chainedvector.jl:664
[6] (::Arrow.var"#107#113"{Vector{Any}, Arrow.Table})(i::Int64)
@ Arrow ~/.julia/packages/Arrow/P0wVk/src/table.jl:313
[7] foreach(f::Arrow.var"#107#113"{Vector{Any}, Arrow.Table}, itr::UnitRange{Int64})
@ Base ./abstractarray.jl:3073
[8] macro expansion
@ ~/.julia/packages/Arrow/P0wVk/src/table.jl:312 [inlined]
[9] (::Arrow.var"#104#110"{Channel{Any}, Arrow.Table})()
@ Arrow ./threadingconstructs.jl:341
```
I've reproduced this in both `1.8.5` and `1.9.0-rc2`, in two machines.
Machine 1:
```
julia> versioninfo()
Julia Version 1.9.0-rc2
Commit 72aec423c2a (2023-04-01 10:41 UTC)
Platform Info:
OS: Linux (x86_64-linux-gnu)
CPU: 8 × Intel(R) Core(TM) i7-8665U CPU @ 1.90GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-14.0.6 (ORCJIT, skylake)
Threads: 10 on 8 virtual cores
Environment:
JULIA_EDITOR = code
JULIA_NUM_THREADS =
```
Machine 2:
```
julia> versioninfo()
Julia Version 1.8.5
Commit 17cfb8e65ea (2023-01-08 06:45 UTC)
Platform Info:
OS: Linux (x86_64-linux-gnu)
CPU: 48 × Intel(R) Xeon(R) Gold 6136 CPU @ 3.00GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-13.0.1 (ORCJIT, skylake-avx512)
Threads: 20 on 48 virtual cores
Environment:
JULIA_EDITOR = code
JULIA_NUM_THREADS = 8
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-julia] ericphanson commented on issue #420: Flaky test
Posted by "ericphanson (via GitHub)" <gi...@apache.org>.
ericphanson commented on issue #420:
URL: https://github.com/apache/arrow-julia/issues/420#issuecomment-1501112216
That sounds like a bug that the test is catching rather than just a flaky test!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org