You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "raulcd (via GitHub)" <gi...@apache.org> on 2024/04/04 12:36:26 UTC
[I] [CI][C++] CPP valgrind job is failing on Azure tests [arrow]
raulcd opened a new issue, #41004:
URL: https://github.com/apache/arrow/issues/41004
### Describe the bug, including details regarding any error messages, version, and platform.
[test-conda-cpp-valgrind](https://github.com/ursacomputing/crossbow/runs/23421876917) is failing on `TestAzureFileSystenOnAllEnv` and `TestAzureFileSystemOnAllScenarios` test failures:
```
[ FAILED ] 5 tests, listed below:
[ FAILED ] TestAzureFileSystemOnAllEnvs/0.CreateDirOnRoot, where TypeParam = arrow::fs::TestingScenario<arrow::fs::AzuriteEnv, false>
[ FAILED ] TestAzureFileSystemOnAllScenarios/0.CreateDirOnMissingContainer, where TypeParam = arrow::fs::TestingScenario<arrow::fs::AzuriteEnv, true>
[ FAILED ] TestAzureFileSystemOnAllScenarios/0.OpenOutputStreamWithMissingContainer, where TypeParam = arrow::fs::TestingScenario<arrow::fs::AzuriteEnv, true>
[ FAILED ] TestAzureFileSystemOnAllScenarios/1.CreateDirOnMissingContainer, where TypeParam = arrow::fs::TestingScenario<arrow::fs::AzuriteEnv, false>
[ FAILED ] TestAzureFileSystemOnAllScenarios/1.OpenOutputStreamWithMissingContainer, where TypeParam = arrow::fs::TestingScenario<arrow::fs::AzuriteEnv, false>
5 FAILED TESTS
==16391==
==16391== HEAP SUMMARY:
==16391== in use at exit: 11,298 bytes in 159 blocks
==16391== total heap usage: 3,136,856 allocs, 3,136,697 frees, 5,688,532,534 bytes allocated
==16391==
==16391== 1,424 bytes in 2 blocks are definitely lost in loss record 19 of 20
==16391== at 0x485094F: calloc (vg_replace_malloc.c:1595)
==16391== by 0x7EA454F: xmlGetThreadLocalStorage (in /opt/conda/envs/arrow/lib/libxml2.so.2.12.6)
==16391== by 0x7EA52DD: __xmlDefaultBufferSize (in /opt/conda/envs/arrow/lib/libxml2.so.2.12.6)
==16391== by 0x7EE149A: xmlAllocParserInputBuffer (in /opt/conda/envs/arrow/lib/libxml2.so.2.12.6)
==16391== by 0x7EE1ACA: xmlParserInputBufferCreateStatic (in /opt/conda/envs/arrow/lib/libxml2.so.2.12.6)
==16391== by 0x7EE1B6D: xmlParserInputBufferCreateMem (in /opt/conda/envs/arrow/lib/libxml2.so.2.12.6)
==16391== by 0x7F12AF8: xmlReaderForMemory (in /opt/conda/envs/arrow/lib/libxml2.so.2.12.6)
==16391== by 0x6BB3363: Azure::Storage::_internal::XmlReader::XmlReader(char const*, unsigned long) (in /opt/conda/envs/arrow/lib/libazure-storage-common.so)
==16391== by 0x4B8B690: Azure::Storage::Blobs::_detail::BlobContainerClient::ListBlobsByHierarchy(Azure::Core::Http::_internal::HttpPipeline&, Azure::Core::Url const&, Azure::Storage::Blobs::_detail::BlobContainerClient::ListBlobContainerBlobsByHierarchyOptions const&, Azure::Core::Context const&) (in /opt/conda/envs/arrow/lib/libazure-storage-blobs.so)
==16391== by 0x4AD5185: Azure::Storage::Blobs::BlobContainerClient::ListBlobsByHierarchy(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, Azure::Storage::Blobs::ListBlobsOptions const&, Azure::Core::Context const&) const (in /opt/conda/envs/arrow/lib/libazure-storage-blobs.so)
==16391== by 0x5CE0878: arrow::fs::AzureFileSystem::Impl::GetFileInfoWithSelectorFromContainer(Azure::Storage::Blobs::BlobContainerClient const&, Azure::Core::Context const&, Azure::Nullable<int>, arrow::fs::FileSelector const&, std::vector<arrow::fs::FileInfo, std::allocator<arrow::fs::FileInfo> >*) (azurefs.cc:1680)
==16391== by 0x5CE0E6A: arrow::fs::AzureFileSystem::Impl::GetFileInfoWithSelector(Azure::Core::Context const&, Azure::Nullable<int>, arrow::fs::FileSelector const&, std::vector<arrow::fs::FileInfo, std::allocator<arrow::fs::FileInfo> >*) (azurefs.cc:1778)
==16391== by 0x5CE0F73: arrow::fs::AzureFileSystem::GetFileInfo(arrow::fs::FileSelector const&) (azurefs.cc:2863)
==16391== by 0x5C8C2B0: operator() (filesystem.cc:182)
==16391== by 0x5C8C2B0: std::enable_if<((!std::is_void<arrow::Result<std::vector<arrow::fs::FileInfo, std::allocator<arrow::fs::FileInfo> > > >::value)&&(!arrow::detail::is_future<arrow::Result<std::vector<arrow::fs::FileInfo, std::allocator<arrow::fs::FileInfo> > > >::value))&&((!arrow::Future<std::vector<arrow::fs::FileInfo, std::allocator<arrow::fs::FileInfo> > >::is_empty)||std::is_same<arrow::Result<std::vector<arrow::fs::FileInfo, std::allocator<arrow::fs::FileInfo> > >, arrow::Status>::value), void>::type arrow::detail::ContinueFuture::operator()<arrow::fs::FileSystem::GetFileInfoGenerator(arrow::fs::FileSelector const&)::{lambda(std::shared_ptr<arrow::fs::FileSystem>)#1}&, std::shared_ptr<arrow::fs::FileSystem>&, arrow::Result<std::vector<arrow::fs::FileInfo, std::allocator<arrow::fs::FileInfo> > >, arrow::Future<std::vector<arrow::fs::FileInfo, std::allocator<arrow::fs::FileInfo> > > >(arrow::Future<std::vector<arrow::fs::FileInfo, std::allocator<arrow::fs::FileInfo> >
>, arrow::fs::FileSystem::GetFileInfoGenerator(arrow::fs::FileSelector const&)::{lambda(std::shared_ptr<arrow::fs::FileSystem>)#1}&, std::shared_ptr<arrow::fs::FileSystem>&) const (future.h:150)
==16391== by 0x5C8C369: void std::__invoke_impl<void, arrow::detail::ContinueFuture&, arrow::Future<std::vector<arrow::fs::FileInfo, std::allocator<arrow::fs::FileInfo> > >&, arrow::fs::FileSystem::GetFileInfoGenerator(arrow::fs::FileSelector const&)::{lambda(std::shared_ptr<arrow::fs::FileSystem>)#1}&, std::shared_ptr<arrow::fs::FileSystem>&>(std::__invoke_other, arrow::detail::ContinueFuture&, arrow::Future<std::vector<arrow::fs::FileInfo, std::allocator<arrow::fs::FileInfo> > >&, arrow::fs::FileSystem::GetFileInfoGenerator(arrow::fs::FileSelector const&)::{lambda(std::shared_ptr<arrow::fs::FileSystem>)#1}&, std::shared_ptr<arrow::fs::FileSystem>&) (invoke.h:61)
==16391== by 0x5C8C3C8: __invoke<arrow::detail::ContinueFuture&, arrow::Future<std::vector<arrow::fs::FileInfo, std::allocator<arrow::fs::FileInfo> > >&, arrow::fs::FileSystem::GetFileInfoGenerator(const arrow::fs::FileSelector&)::<lambda(std::shared_ptr<arrow::fs::FileSystem>)>&, std::shared_ptr<arrow::fs::FileSystem>&> (invoke.h:96)
==16391== by 0x5C8C3C8: __call<void, 0, 1, 2> (functional:495)
==16391== by 0x5C8C3C8: void std::_Bind<arrow::detail::ContinueFuture (arrow::Future<std::vector<arrow::fs::FileInfo, std::allocator<arrow::fs::FileInfo> > >, arrow::fs::FileSystem::GetFileInfoGenerator(arrow::fs::FileSelector const&)::{lambda(std::shared_ptr<arrow::fs::FileSystem>)#1}, std::shared_ptr<arrow::fs::FileSystem>)>::operator()<, void>() (functional:580)
==16391== by 0x5C8C3DA: arrow::internal::FnOnce<void ()>::FnImpl<std::_Bind<arrow::detail::ContinueFuture (arrow::Future<std::vector<arrow::fs::FileInfo, std::allocator<arrow::fs::FileInfo> > >, arrow::fs::FileSystem::GetFileInfoGenerator(arrow::fs::FileSelector const&)::{lambda(std::shared_ptr<arrow::fs::FileSystem>)#1}, std::shared_ptr<arrow::fs::FileSystem>)> >::invoke() (functional.h:152)
==16391== by 0x5ECA64B: arrow::internal::FnOnce<void ()>::operator()() && (functional.h:140)
==16391== by 0x5ECCEC0: arrow::internal::WorkerLoop(std::shared_ptr<arrow::internal::ThreadPool::State>, std::_List_iterator<std::thread>) (thread_pool.cc:457)
==16391== by 0x5ECD199: arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::{lambda()#1}::operator()() const (thread_pool.cc:618)
==16391== by 0x5ECD1EE: __invoke_impl<void, arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::<lambda()> > (invoke.h:61)
==16391== by 0x5ECD1EE: __invoke<arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::<lambda()> > (invoke.h:96)
==16391== by 0x5ECD1EE: _M_invoke<0> (std_thread.h:279)
==16391== by 0x5ECD1EE: operator() (std_thread.h:286)
==16391== by 0x5ECD1EE: std::thread::_State_impl<std::thread::_Invoker<std::tuple<arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::{lambda()#1}> > >::_M_run() (std_thread.h:231)
==16391== by 0x6731E94: execute_native_thread_routine (thread.cc:104)
==16391== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==16391==
==16391== For lists of detected and suppressed errors, rerun with: -s
==16391== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
```
### Component(s)
C++, Continuous Integration
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [CI][C++] CPP valgrind job is failing on Azure tests [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #41004:
URL: https://github.com/apache/arrow/issues/41004#issuecomment-2039996731
How about trying this on local as the first step?
I think that we need to know why full response isn't returned.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [CI][C++] CPP valgrind job is failing on Azure tests [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #41004:
URL: https://github.com/apache/arrow/issues/41004#issuecomment-2037151359
The Valgrind may be unrelated. Invalid response from Azurite may be related instead:
https://dev.azure.com/ursacomputing/crossbow/_build/results?buildId=62773&view=logs&j=50a69d0a-7972-5459-cdae-135ee6ebe312&t=13df7b5c-76db-5c26-6592-75581a9ed64a&l=8681
```text
[ RUN ] TestAzureFileSystemOnAllEnvs/0.CreateDirOnRoot
unknown file: Failure
C++ exception with description "Connection closed before getting full response or response is less than expected. Expected response length = 254. Read until now = 236" thrown in the test body.
```
BTW, the Valgrind report may be related to https://github.com/apache/arrow/pull/40567/files#diff-7d2cbf9c6abf1ee980b8cf5a79e222543c17b3d93b12c4c0c9fa123b1bb200b6R398-R412 .
`GetFileInfoGenerator()` may report the report.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [CI][C++] CPP valgrind job is failing on Azure tests [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #41004:
URL: https://github.com/apache/arrow/issues/41004#issuecomment-2050861171
Issue resolved by pull request 41163
https://github.com/apache/arrow/pull/41163
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [CI][C++] CPP valgrind job is failing on Azure tests [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #41004:
URL: https://github.com/apache/arrow/issues/41004#issuecomment-2049977409
This is probably not a regression, so not a blocker for 16.0.0?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [CI][C++] CPP valgrind job is failing on Azure tests [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #41004:
URL: https://github.com/apache/arrow/issues/41004#issuecomment-2050861764
@raulcd You can include this to 16.0.0 or move this to 17.0.0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [CI][C++] CPP valgrind job is failing on Azure tests [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou closed issue #41004: [CI][C++] CPP valgrind job is failing on Azure tests
URL: https://github.com/apache/arrow/issues/41004
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [CI][C++] CPP valgrind job is failing on Azure tests [arrow]
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on issue #41004:
URL: https://github.com/apache/arrow/issues/41004#issuecomment-2037081482
cc @felipecrv
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [CI][C++] CPP valgrind job is failing on Azure tests [arrow]
Posted by "felipecrv (via GitHub)" <gi...@apache.org>.
felipecrv commented on issue #41004:
URL: https://github.com/apache/arrow/issues/41004#issuecomment-2039935566
@kou but how to proceed here, then? Should we add a try-catch in the code to guard against `azurite` behavior or disable the test case?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org